Shing Lyu - Articles

Lessons learned in writing my first book

Sun, 05 Apr 2020 08:42:16 +0000

You might have noticed that I didn’t update this blog frequently in the past year. It’s not because I’m lazy, but I focused all my creative energy on writing this book: Practical Rust Projects. The book is now available on Apress, Amazon and O’Reilly. In this post, I’ll share some of the lessons I learned in writing this book.

Although I’ve been writing Rust for quite a few years, I haven’t really studied the internals of the Rust language itself. Many of the Rust enthusiasts whom I know seem to be having much fun appreciating how the language is designed and built. But I take more joy in using the language to build tangible things. Therefore, I’ve been thinking about writing a cookbook-style book on how to build practical projects with Rust, ever since I finished the video course Building Reusable Code with Rust.

Out of my surprise, I received an email from Steve Anglin, an acquisition editor from Apress, in April 2019. He initially asked me to write a book on the RustPython project. But the project was still growing rapidly thanks to the contributors. I’ve already lost grip on the overall architecture, so I can’t really write much about it. So I proposed the topic I have in mind to Steve. Fortunately, the editorial board accepted my proposal, and we decided to write two books: one for general Rust projects and one for web-related Rust projects.

Since this is my first time writing a book that will be published in physical form (or as The Rust Book put it, “dead tree form”), I learned quite a lot throughout the process. Hopefully, these points will help you if you are considering or are already writing your own book.

The Pomodoro Technique works!

It might be tempting to write a waterfall-style book writing plan, something along the lines of “I’ll write one section per day”. But there is a fundamental flaw with this approach: not all sections are of the same length and difficulty. A section about how to setup software might be very mechanical and easy to write, but a section about the history of a piece of software might take you days of research. Therefore, you can’t reliably predict how long it will take to finish a chapter, resulting in fear of starting and procrastination.

I find the Pomodoro Technique to be a constructive alternative in structuring my writing plan. The Pomodoro Technique is a time management method where you break your work into 25 minutes intervals and rest in between. Instead of saying, “I’ll finish this chapter by today”, say “I’ll do 2 Pomodoro sessions (25 x 2 = 50 min) today”. This way, no matter if you are making good progress or not, you know you are committing enough effort into the book writing project. A good side effect is that once I started writing, I got into the flow and ended up writing more Pomodoro sessions then I planned.

An Outline is important

I usually write freely without and outline when I write my blog. But writing a book is a completely different thing. An outline will help you structure the content much better and avoid missing important topics. I usually start with a very high-level chapter outline, then add a section outline before I start writing each section. It’s nice to write the outline for all the chapters before starting. When you are writing Chapter 1 and suddenly have an idea about something in Chapter 3, you can quickly add that to the Chapter 3 outline as a reminder.

Example code is a crucial part of a programming book. It’s also beneficial to list down each step in the outline while you develop the example code. Because if you write all the example code in one go, you’ll forget about many steps that are not in the final code when you revisit it. For example,

Installing an external dependency
Important trail-and-error you went through
Informative compiler errors and warnings

These are all key points that need to be mentioned, but they don’t show up in the final code. Although you can dig that up by going through the git history, it’s still easier to write them down during development.

Version control and LaTex

Version control is not only for code. The draft will go through multiple reviews and revise passes. You’ll usually have to revise the previous chapter while you are writing the next one. So use a format that is diff-friendly is very helpful. I use an unofficial LaTex template provided by Apress, with my own modifications. There are some random LaTex tips:

A professional production team lays out my page for softcover prints. So the image positioning from LaTex will not be the final one. But it’s still crucial to place the image after they are referenced. So I sometimes need to use \clearpage to force the image to appear on the next page.
I render the draft on A4 paper, but the final book is narrower and has a 65 character limit on the source code line length. Therefore, the production team has to wrap the code lines in many places. But since they are not Rust experts, they sometimes wrap the line in a non-idiomatic way. So it’s worth checking with your editor what is the source code width limit.
Take advantage of shell scripts to help you with writing. I created scripts for:
- Scan the LaTex code to find missing code listing files or images.
- Linting the code examples.
- Linting common grammatical or style errors.

Use a grammar checker, multiple times

I’m not a native English speaker, so I use aspell and Grammarly to check my grammar (disclaimer: Grammarly does not sponsor me). GNU Aspell is an open-source spell checker for the command line. Although it mostly checks for spelling, not grammar, its command-line interface and keyboard control is much more efficient than clicking the mouse. So I use Aspell to fix apparent typos. Then I copy-paste the LaTex source code into Grammarly’s web interface for a more thorough grammar check. However, many of the latex annotations like \texttt{code formatting} breaks the sentence, so Grammarly misses some sentences. So after the first round of Grammarly check, I render the LaTex code into PDF and use the pdftotext command to extract the rendered text from the PDF. I then run Grammarly through this text again.

Of course, to fix this problem once and for all, improving the writing skill is critical. I came across this Google Technical Writing Course after I finish this book, which helped me a lot in the writing process of my second book.

Get the book now

In this post, I focused on a few practical tips about book writing. I can maybe write another post about my LaTex setup and my observation about the Rust ecosystem. Leave a message using the “Message me” at the bottom right to let me know if that will be interesting to you.

Please grab a copy of the book at the book store of your choice and let me know what you think:

Download JavaScript Data as Files on the Client Side

Sat, 09 Feb 2019 09:28:39 +0000

When building websites or web apps, creating a “Download as file” link is quite useful. For example if you want to allow user to export some data as JSON, CSV or plain text files so they can open them in external programs or load them back later. Usually this requires a web server to format the file and serve it. But actually you can export arbitrary JavaScript variable to file entirely on the client side. I have implemented that function in one of my project, MozApoy, and here I’ll explain how I did that.

First, we create a link in HTML

<a id="download_link" download="my_exported_file.txt" href=”” >Download as Text File</a>

The download attribute will be the filename for your file. It will look like this:

Notice that we keep the href attribute blank. Traditionally we fill this attribute with a server-generated file path, but this time we’ll assign it dynamically generate the link using JavaScript.

Then, if we want to export the content of the text variable as a text file, we can use this JavaScript code:

var text = 'Some data I want to export';
var data = new Blob([text], {type: 'text/plain'});

var url = window.URL.createObjectURL(data);

document.getElementById('download_link').href = url;

The magic happens on the third line, the window.URL.createObjectURL() API takes a Blob and returns an URL to access it. The URL lives as long as the document in the window on which it was created. Notice that you can assign the type of the data in the new Blob() constructor. If you assign the correct format, the browser can better handle the file. Other commonly seen formats include application/json and text/csv. For example, if we name the file as *.csv and give it type: 'text/csv', Firefox will recognize it as “CSV document” and suggest you open it with LibreOffice Calc.

And in the last line we assign the url to the <a/> element’s href attribute, so when the user clicks on the link, the browser will initiate an download action (or other default action for the specific file type.)

Everytime you call createObjectURL(), a new object URL will be created, which will use up the memory if you call it many times. So if you don’t need the old URL anymore, you should call the revokeObjectURL() API to free them.

var url = window.URL.createObjectURL(data);
window.URL.revokeObjectURL(url);

This is a simple trick to let your user download files without setting up any server. If you want to see it in action, you can check out this CodePen.

Counting your contribution to a git repository

Tue, 25 Dec 2018 11:00:27 +0000

Sometimes you may wonder, how many commits or lines of code did I contributed to a git repository? Here are some easy one liners to help you count that.

Number of commits

Let’s start with the easy one: counting the number of commits made by one user.

The easiest way is to run

git shortlog -s

This gives you a list of commit counts by user:

Grant Lindberg
Jonathan Hao
Matias Kinnunen
Shing Lyu
Shou Ya
wildsky
wildskyf

(The example comes from shinglyu/QuantumVim.)

If you only care about one user you can use

git rev-list HEAD --author="Shing Lyu" --count 

, which prints 65.

Let’s explain how this works:

git rev-list HEAD will list the commit objects in HEAD
--author="Shing Lyu" will filter out only the commits made by the author Shing Lyu
--count counts the number of commits. You can pipe it to | wc -l instead.

Count the line of insertion and deletions by a user

Insertion and deletions are a little bit tricker. This is what I came up with:

git log --author=Shing --pretty=tformat: --numstat | grep -v '^-' | awk '{ add+=$1; remove+=$2 } END { print add, remove }' 

This might seem a little bit daunting, but we’ll break it up into steps:

git log --author="Shing Lyu" list the commits by Shing Lyu, in the following format:

commit 6966b2c969cbf62029792221bf124ed75ee2c640
Author: Shing Lyu 
Date:   Sat Nov 18 17:01:25 2017 +0100

    Added Ctrl+z to close all system tabs

commit f4710cc3a2efdc63c7caf3ec04d504912ad20a93
Author: Shing Lyu 
Date:   Sat Nov 18 15:58:20 2017 +0100

    Bump version and diable jpm packaging

--numstat will give us the line added and removed per file per commit:

commit 6966b2c969cbf62029792221bf124ed75ee2c640
Author: Shing Lyu 
Date:   Sat Nov 18 17:01:25 2017 +0100

    Added Ctrl+z to close all system tabs

    1       0       README.md
    10      0       manifest.json
    6       1       package.sh
    35      0       vim-background.js
    4       1       vim.js

commit f4710cc3a2efdc63c7caf3ec04d504912ad20a93
Author: Shing Lyu 
Date:   Sat Nov 18 15:58:20 2017 +0100

    Bump version and diable jpm packaging

    1       1       manifest.json
    3       3       package.sh

We don’t really need the commit, Author, Date and commit message fields, so we use an empty formatting string to get rid of them: --pretty=tformat:

     0       README.md
    0       manifest.json
     1       package.sh
    0       vim-background.js
     1       vim.js
     1       manifest.json
     3       package.sh

If you add some non-text files, e.g. png image files, the insertion/deletion count might be represented as - - foo.png. Therefore we filter them out with grep -v '^-'. If you are not familiar with grep, -v means reverse match (i.e. find those lines that does NOT match the patter). The pattern ^- means lines staring with a -. (This part is optional if you pipe to awk, awk seems to ignore non-numeric character while doing the math.)
Finally we pipe it to awk for summing. Even if you are not familiar with awk, this part is pretty self-explanatory:
```
awk '{ add+=$1; remove+=$2 } END { print add, remove }'
```
We add column one ($1) to the variable add, and column two ($2) to the variable remove, then we print them out. This gives us an output like so:
```
936 260 
```

Other alternatives

There are many other off-the-shelf scrips that will help you calculate contribution statistics. Like git-quick-stats, git-fame and git-fame-rb. But if you only want a quick-and-easy solution please give it a try.

New Rust Course - Building Reuseable Code with Rust

Fri, 16 Nov 2018 21:12:18 +0000

My first ever video course is now live on Udemy, Safari Books and Packt. It really took me a long time and I’d love to share with you what I’ve prepared for you.

What’s this course about?

This course is about the Rust programming language, but it’s not those general introductory course on basic Rust syntax. This course focus on the code reuse aspect of the Rust language. So we won’t be touch every language feature, but we’ll help you understand how a selected set of features will help you achieve code reuse.

What’s so special about it?

Since these course is not a general introduction course, it is structured in a way that is bottom-up and help you learn how the features are actually used out in the wild.

A bottom up approach

We started from the most basic programming language construct: loops, iterators and functions. Then we see how we can further generalize functions and data structures (structs and enums) using generics. With these tools, we can avoid copy-pasting and stick to the DRY (Don’t Repeat Yourself) principle.

But simply avoiding repeated code snippet is not enough. What comes next naturally is to define a clear interface, or internal API between the modules (in a general sense, not the Rust mod). This is when traits comes in handy. Traits help you define and enforce interfaces. We’ll also discuss the performance impact on static dispatch vs. dynamic dispatch by using generics and trait object.

Finally we talk about more advanced (i.e. you shouldn’t use it unless necessary) tool like macros, which will help do crazier things by tapping directly into the compiler. You can write function-like macros that can help you reuse code that needs lower level access. You can also create custom derive with macros.

Finally, with these tools at hand, we can package our code into modules (Rust mod), which can help you define a hierarchical namespace. We can then organize these modules into Crates, which are software packages or libraries that can contain multiple files. When you reach this level, you can already consume and produce libraries and frameworks and work with a team of Rust developers.

A guided tour though the `std`

It’s very easy to learn a lot of syntax, but never understand how they are used in real life. In each section, we’ll guide you through how these programming tools are used in std, or the Rust standard library. Standard libraries are the extreme form of code reuse, you are reusing code that is produced by the core language team. You’ll be able to see how these features are put to real use in std to solve their code reuse needs.

We’ll also show you how you can publish your code onto crates.io, Rust’s package registry. Therefore you’ll not only be comfortable reusing other people’s crate, but be a valuable contributor to the wider Rust community.

Conclusion

So this summarized the highlights of this course. If you’ve already learned the basics of Rust and would like to take your Rust skill to the next level, please check this course out. You can find this course on the following platforms: Udemy, Safari Books and Packt.

Chatting with your website visitors through Chatra

Mon, 13 Aug 2018 18:59:29 +0000

Update (May 31, 2025): I’ve removed the Chatra chat widget from my website today. While it was a useful tool, I found that many users mistakenly thought it was AI-powered and would be quite rude when interacting with what they assumed was a chatbot. If you need to reach out, please feel free to contact me via email listed at the bottom of this website instead.

When I started the blog, I didn’t add a message board below each article because I don’t have the time to deal with spam. Due to broken windows theory, if I leave the spam unattended my blog will soon become a landfill for spammers. But nowadays many e-commerce site or brand sites have a live chatting box, which will solve my problem because I can simply ignore spam, while interested readers can ask questions and provide feedbacks easily. That’s why when my sponsor, Chatra.io, approached me with their great tool, I fell in love with it right away and must share it with everyone.

How it works

First, signup for a free account here, and you’ll be logged into a clean and modern chat interface.

You’ll get a JavaScript widget snippet (in the “Set up & customize” page or email), which you can easily place onto your site (even if you don’t have a backend, like this site). A chat button will immediately appear on your site. Your visitor can now send you messages, and you can choose to reply them right away or followup later using their web dashboard, desktop or mobile app.

What I love about Chatra

Easy setup and clean UI

As you can see, the setup is simply pasting a block of code into your blog template (or use their app or plugin for your platform), and it works right away. The chat interface is modern and clean, you can “get it” within no time if you ever used any chat app.

Considerations for bloggers who can’t be online all day

You might wonder, “I don’t have an army of customer service agents, how can I keep everyone happy with only myself replying messages?”. But Chatra already considered that for you with messenger mode, which can receive messages 24/7 even if you are offline. A bot will automatically reply to your visitor and ask for their contact details, so you can follow up later with an email. Every live or missed message can be configured to be sent to your email, so you can check them in batch after a while. Also messaging history are preserved even if the visitor left and come back later, so you get the context of what they were saying. It’s also important to to set expectations for your visitor, to let them know you are working alone and can’t reply super fast. That brings us to the next point: customizable welcome messages and prompts.

Customizable and programmable

Almost everything in Chatra is customizable. From the welcome message, chat button text, to the automatic reply content. So instead of saying “We are a big team and we’ll reply in 10 mins, guaranteed!”, you can instead say something along the line of “Hi, I’m running this site alone and I’d love to hear from you. I’ll get back to you within days”. Besides customizing the look and feel and tone of speech, you can also setup triggers that automatically initiate a chat when criteria meet. For example we can send a automated message when a visitor reads the article for more then 1 minute. Of course you can further customize the experience using the developer API.

Out-of-the-box Google Analytics integration

One thing I really care about is understanding how my visitors interact with the site, and how I can optimize the content and UX to further engage them. I did that through Google Analytic. Much to my amazement, Chatra detected my Google analytics configuration and automatically send relevant events to my Google Analytic tracking, without me even setting up anything. I can directly create goals based on the events and track the conversion funnel leading to a chat.

Pricing and features

Chatra has a free and a paid plan, and also a 2-week trial period that allows you to test everything before paying. The number of registered agents, connected websites and concurrent chats is unlimited in all plans. The free plan has basic features, including mobile apps, Google Analytics integration, some API options, etc., and allows 1 agent to be online at a time, which is sufficient enough for a one-person website like mine. But you can have several agents taking turns chatting: when one goes offline, another connects. And even if the online spot is already taken, other agents can still access the dashboard and read chats.

The paid plan starts at $15 per month and gives you access to all features, including automatic triggers and visitors online list, saved replies, typing insights, visitor information, integration with services like Zapier, Slack, Help Scout and more, and allows as many agents online as paid for. Agents on the paid plan can also take turns chatting, so there’s no need to pay for all of them.

Conclusion

All in all, Chatra is a nice tool to further engage your visitors. The free plan is generous enough for most small scale websites. In case you scales up in the future, their paid plan is affordable and pays for itself after a few successful sales. So if you want an easy and convenient way to chat with your visitors, gain feedback and have more insights into your users, you should give Chatra a try with this link now.

How to Unit Test WebExtensions

Sun, 24 Jun 2018 19:44:13 +0000

We all know that unit-testing is a good software engineering practice, but sometimes the hassle of setting up the testing environment will keep us from doing it in the first place. After Firefox 57, WebExtension has become the new standard for writing add-ons for Firefox. How do you set up everything to start testing your WebExtension-based add-ons?

In the earlier format of the Firefox add-ons, namely the Add-on SDK (a.k.a. Jetpack), there is a built-in command for unit-test (jpm test). But for WebExtension, as far as I know, doesn’t have such thing built in. Luckily all the technology used in WebExtension is still standard web technology, so we can use off-the-shelf JavaScript unit-testing frameworks.

I want to keep my tests as simple as possible, so I made some assumptions:

I don’t test WebExtension API calls. I keep a thin layer of wrapper around WebExtension API calls, and I don’t put too much logic into them. So hopefully the risk is low enough to not test. Anything more complex like the business logic or custom data structures or functions are all tested.
I don’t like to use non-standard module systems. As far as I know WebExtension doesn’t support ES6 module yet. So I follow the good old way of including all the JavaScript I need in the page (or as a background page).
I don’t use Node.js libraries in add-ons, period.

Mocha and expect.js

We will be using Mocha test framework and expect.js assertion library, but you can use any test framework that supports running in browsers. We’ll be using the browser version of Mocha. You need to create an HTML file like this:

<html>
  <head>
    <meta charset="utf-8">
    <title>Unit Tests (by Mocha)</title>
    <link href="https://cdn.rawgit.com/mochajs/mocha/2.2.5/mocha.css" rel="stylesheet" />
  </head>
  <body>
    <div id="mocha"></div>

    <script src="https://cdn.rawgit.com/Automattic/expect.js/0.3.1/index.js"></script>
    <script src="https://cdn.rawgit.com/mochajs/mocha/2.2.5/mocha.js"></script>

    <script>mocha.setup('bdd')</script>

    <script src="calculator.js"></script>
    <script src="test.calculator.js"></script>

    <script>
      mocha.checkLeaks();
      mocha.run();
    </script>
  </body>
</html>

In the file you can see that we imported the Mocha library and expect.js library from CDN, so we don’t need to install anything locally. We’ll be testing a imaginary calculator library used in our extension. The test cases are written in the test.calculator.js file. The classes and functions under tested are placed in the calculator.js file. We load the module under test and the test case in the file as well

The way to run it is to simply open this file in a Firefox, if everything goes well you should see the following screen:

I usually put the main logic and code that interacts with extension APIs in a file named background.js. Any other business and utility functions goes into separate JS files, which are test using the above unit testing framework. They are all load together with background.js as background scripts. To do so, you need to add all of them to manifest.json like so:

{
  "name": "Calculator Add-on",
  ...
  "background": {
    "scripts": [
      "background.js",
      "calculator.js"
    ]
  },
  ...
}

Now we can write more tests like so in test.calculator.js:

describe('My calculator', function() {
  it('can add 1 and 1 and get 2', function() {
    var result = my_add(1, 1); // my_add is definied in calculator.js
    expect(result).to.eql(2);
  });
});

Run the page again, and you should see it failing, because you haven’t defined my_add yet.

Now, write your my_add in calculator.js:

function my_add(x, y) {
  return x + y;
}

Run the page once again, and your test now passes.

Testing asynchronous code

Many WebExtension APIs return promises so you can write functions that receives and returns promises so you can chain them. The problem is that if you have and error in the Promise chain, the error will be consumed by the promise, so the test framework will not catch it, resulting in an always-passing test. Let’s say you want to write a function that counts how many times you’ve visited Facebook, you can use the browser.hisotry.getVisits() API, which returns a promise. So first we write a test for the count_visits() function we are about to write.

describe('Facebook counter', function() {
   it('can count', function() {
     var mock_getVisits = new Promise(function(resolve, reject) {
       // Simulate the getVisits API that returns 3 results
       resolve([new Object(), new Object(), new Object()]);
     });

     mock_getVisits
       .then(count_visits)
       .then(function(count){
         expect(count).to.eql(3);
       });
  });
});

Then we implement the count_visits(), but we made a typo by writing length as legnht

function count_visits(history_items) {
  return history_items.legnht;
}

In this case, the function will always return undefined, because there is no such thing as legnht for arrays. But if you run the test, you’ll find the test still passing. If you open the developer, you’ll see the actual error. But seems the test framework didn’t catch it.

The latest Mocha library already have built in promise support, you only need to make sure you return the promise so the error can be captured.

describe('Facebook counter', function() {
   it('can count', function() {
     var mock_getVisits = function() {
       return new Promise(function(resolve, reject) {
         // Simulate the getVisits API that returns 3 results
         resolve([new Object(), new Object(), new Object()]);
       });
     };

//   vvvvvv Notice the "return" here
     return mock_getVisits()
       .then(count_visits)
       .then(function(count){
         expect(count).to.eql(3);
       });
  });
});

Now the test fails as expected:

Another way is to use async/await to make your async function looks sync. Mocha can easily handle that as usual synchronous functions.

describe('Facebook counter', function() {
// Notice the async here             vvvvv
   it('can count using async/await', async function() {
     var mock_getVisits = function() {
       return new Promise(function(resolve, reject) {
         // Simulate the getVisits API that returns 3 results
         resolve([new Object(), new Object(), new Object()]);
       });
     }

// and await here vvvvv
     var visits = await mock_getVisits();
     expect(count_visits(visits)).to.eql(3)
  });
});

Conclusion

Writing test WebExtension it’s not as hard as you might think. Simply copying and pasting the HTML and you’re ready to go. You don’t need to install anything or set up complex Node.js compiling pipeline. Start testing your WebExtension code now, it saved me many hours of debugging time, and I believe it will help you as well.

如何貢獻開源專案？

Sat, 12 May 2018 15:55:42 +0000

貢獻開源專案是一個提高自己技能的好機會，同時也能提昇自己在軟體界的能見度與溝通技巧。以下是我貢獻 Mozilla 的 Servo 瀏覽器引擎的一些心得，提供想要一探開源的新手一些方向。

心理準備

首先你要問自己為什麼你想要貢獻開源？是為了提高軟體技術？為了充實履歷表？畢竟能做的專案太多，更別說自己開新專案，所以必須先想清楚自己的初衷，然後找到一個適合自己的需求又能有所貢獻的專案。要找到適合的專案，可以優先從以下方向開始看：

自己正在使用的開源工具/框架/程式語言
自己想學習的新開源技術
自己非常需要的工具，自己開新專案

專案首先一定要對自己有用，否則很難有動力繼續做下去。寫軟體的很大一個動力來自於巧妙的利用程式解決了自己的問題的成就感。如果這個成果又能被其他人使用的話，就是一個更強烈的正向循環。

另外一個常見的錯誤是因為覺得自己技術底子不夠，所以把所有時間花在讀書而沒有動手實做。任何一個程式語言的書絕對都夠你讀個三年五年，更別說每個專案有其背景知識，例如 Web 技術、瀏覽器技術、編譯器、作業系統，即使是大學本科花四年也只能學到皮毛，所以所謂「我先把該學的學完就可以開始貢獻了」絕對是一個癡人說夢的想法。最好的方法是開始從簡單的 bug 下手，例如修一些 typo 或是非常簡單的錯誤，一方面可以練習整個專案的開發流程，也可以開始與專案的其他貢獻者互動。假設程式本身真的太難，也有一些比較簡單的開始方式，例如幫忙撰寫或翻譯技術文件、主辦讀書會、寫部落格分享自己的學習心得、回報軟體的錯誤等等。這些方法可以一邊學習專案本身的技術，同時又開始在社群中培養一些能見度。

最後，開源專案是由人組成的，因此溝通（以及誤解）是逃避不了的。很多時候活躍的專案因為開發速度太快，很少會有完整的文件，很多時候一些架構設計、know-how 可能都在幾個核心成員的腦中。因此當你盡了自己所能讀完了文件與原始碼，還是無法獲得解答的時候，最快的方法就是去專案的聊天室或討論區裡面發問。很多時候你花了幾天讀不懂的原始碼可能只是因為某些歷史原因而這麼複雜，只要簡單一問就能夠馬上進步。不同社群之間的溝通模式差異很大，有些社群內可能核心成員真的很忙，或是比較精英主義，對於新手就會比較冷漠。這時候也只能盡量保持客氣，一般來說只要你的貢獻對社群真的有用，隨著時間過去總是會慢慢被接納。若是社群成員真的太惡意，你大可以找其他的專案貢獻。

以下將以我 2014 年開始貢獻的 Servo 專案作為例子，跟大家介紹一下如何一步一步的開始貢獻。

基礎知識

以下有些基礎技能你最好先學會（或是先掃過一下教學文件然後邊做邊學）：

git: 幾乎所有開源專案都有某種版本控制系統，而 git 是最常見的一種。
- 感謝 Denny 的教學投影片
- 感謝葉闆的教學影片以及Pull request 介紹
GitHub: GitHub 是一個 git 的託管服務，同時他也提供一些協作的功能例如 pull request， issues。熟悉 GitHub 可以幫助你找到數以千計的開源專案，並且快速的加入開發。

找尋可以貢獻的 bug

Servo 的 bug (程式缺陷) 或是新功能請求都在 GitHub 上管理，可以至 Servo 的 Issues 頁面瀏覽。適合新手的 bug 都蒐集在 Servo Starters 頁面上，也可以至 Servo Issues 直接查詢有 E-easy 標籤的 bug。

找到 issue 以後，試著理解問題是什麼，如果下面留言中沒有人表示想要做，就可以立刻留言表示興趣，並且向開 bug 的人請求進一步的指示或協助。以下是一個範例的回應：

Hi @, I'm interested in this issue.

想要知道相關原始碼在哪裡 => Could you point me to the relevant code/file?
想要知道如何重現出這個問題 => Could you tell me the exact step to reproduce this issue?
想要知道相關的標準規範 (W3C, WHATWG, etc. ) => Could point me to the relevant spec?
看不太懂也不知道要問什麼，希望對方進一步說明 => Could you give me more information on how this thing works and how I should proceed?

如何下載、修改並測試 Servo 的原始碼

要取得 Servo 的程式碼開始修改，首先要從 Servo 官方的 repository “fork” 出一版，按下右上方的 “Fork” 按鈕，你的帳號下就會出現一個 servo repository。(詳見Forking)。需要這麼做的原因是因為一般使用者沒有權限直接修改 servo/servo 的原始碼，所以你必須複製(“fork”)一份自己的版本，修改完以後在把改動提交給官方審核 (也就是所謂的 Pull Request，也可以參考這篇文章)。一旦官方對你的 pull request 滿意的話，就會被 merge 到 master，你修改的部份就正式進到 servo 裡面了。

接下來你就可以從你剛剛 fork 出來的 servo repository 按下 “Clone or download”，取得 clone 用的網址。接下來開啟一個終端機，輸入 git clone git@github.com/<你的帳號>/servo.git (把<你的帳號>代換成你的 clone 網址)。這個下載需要耗費不少時間，記得確保你的網路暢通，就可以去泡杯咖啡慢慢等候了。

等下載完畢，首先要做的就是確保你的環境能夠正確的編譯並執行 servo。Servo 的文件中有寫出各個平台所需要事先安裝的一些套件(看這裡) ，例如 Ubuntu 就需要安裝

sudo apt install git curl freeglut3-dev autoconf libx11-dev \
       libfreetype6-dev libgl1-mesa-dri libglib2.0-dev xorg-dev \
       gperf g++ build-essential cmake virtualenv python-pip \
       libssl-dev libbz2-dev libosmesa6-dev libxmu6 libxmu-dev \
       libglu1-mesa-dev libgles2-mesa-dev libegl1-mesa-dev libdbus-1-dev

然後執行 ./mach build --dev 來試著編譯。注意除了 OSX 與 Linux 外，其他平台的狀況都不是很穩定，所以如果無法順利編譯也不要氣餒，記得到 issues 看看有沒有其他人也遇到一樣的問題，或是開個新 issue 尋求協助（記得附上你遇到的錯誤訊息）。

一但編譯順利，你就可以跑 ./mach run 來測試 servo 是否正常運作，你應該會看到 servo 開啟一個視窗，並且載入 servo 的官方網站。你也執行 ./mach run -b 試試看一個用 HTML 打造的簡易瀏覽器界面 browser.html。

如果你編譯一切順利，恭喜！你可以開始動工了。通常我們會為每一個 issue 或是功能開一個新的 branch，以方便切換。當你完成更改以後，記得 commit 並且寫上清楚的 commit 訊息。一般來說好的 commit message 可以寫成：

Implemented <a JavaScript API, CSS property or other feature> (實作了<一個 JavaScript API、CSS 屬性或其他功能>)，例如 Implemented window.open(), 或是 Implemented CSS grid
Modified/Fixed/Changed to (修改/修正/改動了<某個部份>來<提供某些好處>)，例如 Change the internal implementation of class Foo to enhance readabiltiy。

舉例來說，假設我們要實作 window.open() 這個 JavaScript API，我們可以

git branch -b window-open # 開啟一個叫 window-open 的 branch
git checkout window-open
# 實作 window open
./mach build -d  # 確定編譯會過
git commit -m "Implemented window.open()"
git push -u origin window-open # 把 window-open branch 上的東西推到你的 servo repo

如何提交 Pull Request

當你的更改完成，記得先檢查以下幾點，以免浪費其他人的時間：

./mach build -d 可以順利編譯。
./mach test-tidy 會檢查程式風格的問題，例如多餘的空格、錯誤的括號位置、行長度過長之類的問題。
相關的測試有通過（不知道要跑什麼測試的話，請到原始的 issue 中問）:
- ./mach test-unit 單元測試
- ./mach test-wpt web platform test，主要測試 JS/CSS 相關功能是否符合標準規範

如果都通過就可以到 servo/servo，選 Pull requests 然後按 New pull request。接下來按 compare across forks，然後head fork 選擇 <你的帳號>/servo， compare 選擇你要提交的 branch。簡單檢查一下下面出現的程式碼，如果沒有問題就可以按 Create pull request。

當你一開 pull request，servo 的 highfive 聊天機器人就會自動跳出來跟你打招呼，並且幫你指定一個 reviewer。所有提交到 servo 的程式碼都需要通過他人審核，這也是一般 open source 專案的慣例，不過 servo 的 reviewer 都相當友善，所以不用太擔心。

接下來 reviewer 通常會提供你一些修改的建議，這對很多人來說很難為情，因為要把自己寫的程式攤在日光下給人家批評。不過記得 reviewer 都是對事不對人，所以不要把這些評論當作對自己的人身攻擊。這也是一些非常好的學習機會，通常都可以從 reviewer 那邊學到相當多 servo 內部的知識以及程式風格的建議。

如果要根據 reviewer 的意見修正，只要繼續在自己的那份 servo 上修改、commit、並且 git push 到你自己原來的那個 branch 即可，過給秒鐘後再去重新整理 pull request 的頁面就可以發現已經更新到你新推的程式碼了。經過這樣幾次來回修改直到 reviewer 滿意以後，他可能會做幾件事情：

請你 rebase 或 squash，這些操作比較進階，也超出本篇的篇幅，詳細可以參考這篇手把手教學。
他向另外一隻聊天機器人 @bors-servo 說 “@bors-servo try”，這表示他請 bors-servo 啟動一系列自動測試，但是並不直接 merge 你的 pull request。最後如果測試都通過通常就會進到下一條。
他向 bors-servo 說 “@bors-servo r+”，這時候 bors-servo 一樣會先執行一系列測試，如果測試都通過就會直接 merge 你的 pull request。

假設有任何測試沒通過，請不要驚慌，根據測試失敗的原因繼續修改你的程式即可。如果看不懂為何會錯也可以直接在 pull request 裡面發問。

以上流程可以看一個例子：style: Simplify border-image-repeat serialization，從中你可以看到各種機器人如何與 reviewer 合作進行審查與測試。以上就是完整的從找到問題到成功 merge 一個 pull request 的流程，但是在這過程中有許許多多的小環節都有可能出錯，這時千萬記得隨時發問。

哪裡查資料與問問題

Servo 是一個非常複雜而且龐大的專案，而且瀏覽器引擎相關的資料在網路上相當稀少。所以以下我蒐集了一些資源可以幫助你更了解相關的背景知識：

How browsers work: 這是目前網路上最完整最全面的瀏覽器引擎介紹，雖然沒有直接介紹 Servo，但是可以幫助你了解瀏覽器引擎大致是如何運作的。
Matt’s Let’s build a browser engine: Matt Brubeck (也是 Servo 的核心成員) 用實際的 code 介紹如何寫出真正可以運作的瀏覽器引擎。
Servo Wiki: Servo 官方的 wiki，有一些設計的筆記與投影片，不過似乎有一陣子沒更新，記得要檢查一下是不是太舊。
Servo code 裡面的註解與文件: Servo 的程式碼中夾雜了不少註解與文件，假設你在修改某個檔案，記得在同一個資料夾或上下一兩層資料夾找找文件。
WHATWG Spec & W3C Spec: 假設你處理的是 HTML， JavaScript 或是 CSS 的功能，免不了需要查詢一下官方的規範。
MDN: 官方規範太硬太難唸，想要看一下實際上如何使用可以去 MDN 上搜尋。

Servo 的主要成員會定期的巡視 GitHub 上的 issue 與 pull request，所以有 issue 相關的問題可以直接在 GitHub 上發問。另外也可以到 #servo IRC 上直接問問題，通常可以更快得到答案。

其他人的貢獻經驗

網路上也有不少人分享他們貢獻 Servo 的經驗，例如來自台灣的劉安齊在 Medium 上分享過「踏入 Mozilla Servo 兩個月的心得」。也可以看看 Fausto NA 的 “My experience contributing to Servo“。開源是一趟冒險，每個人的體驗都會不同，這裡只能介紹一些比較技術性的面向。希望各位都能在這個過程中學到自己想學的，同時又能對社群有所貢獻。

Merge Pull Requests without Merge Commits

Sun, 25 Mar 2018 21:46:36 +0000

By default, GitHub’s pull request (or GitLab’s merge request) will merge with a merge commit. That means your feature branch will be merged into the master by creating a new commit, and both the feature and master branch will be kept.

Let’s illustrate with an example:

Let’s assume we branch out a feature branch called “new-feature” from the master branch, and pushed a commit called “Finished my new feature”. At the same time someone pushed another commit called “Other’s feature” onto the master branch.

If we now create a pull request for our branch, and get merged, we’ll see a new merge commit called “Merge branch ‘new-feature’”

If you look at GitHub’s commit history, you’ll notice that the UI shows a linear history, and the commits are ordered by the time they were pushed. So if multiple people merged multiple branches, all of them will be mixed up. The commits on your branch might interlace with other people’s commits. More importantly, some development teams don’t use pull request or merge requests at all. Everyone is suppose to push directly to master, and maintain a linear history. How can you develop in branches but merge them back to master without a merge commit?

Under the hood, GitHub and GitLab’s “merge” button uses the --no-ff option, which will force create a merge commit. What you are looking for is the opposite: --ff-only (ff stands for fast-forward). This option will cleanly append your commits to master, without creating a merge commit. But it only works if there is not new commits in master but not in your feature branch, otherwise it will fail with a warning. So if someone pushes to master and you did a git pull on your local master, you need to do a rebase on your feature branch before using --ff-only merge. Let’s see how to do this with an example:

git checkout new-feature # Go to the feature branch named "new-feature"
git rebase master
# Now your feature have all the commits from master
git checkout master #Go back to master
git merge --ff-only new-feature

After these commands, your master branch should contain the commits from the feature branch, as if they are cherry-picked from the feature branch. You can then push directly remote.

git push

If unfortunately someone pushed more code to the remote master while you are doing this, your push might fail. You can pull, rebase and push again like so:

git pull --rebase && git push

GitHub’s documentation has some nice illustrations about the two different kind of merges.

Here is a script that does the above for you. To run it you have to checkout to the feature branch you want to merge back to master, then execute it. It will also pull and rebase both your feature and master branch to the most up-to-date remote master during the operation.

#!/usr/bin/env bash
CURRBRANCH=$(git rev-parse --abbrev-ref HEAD)
if [ $CURRBRANCH = "master" ]
then
    echo "Already on master, aborting..."
    exit 1 
fi

echo "Merging the change from $CURRBRANCH to master..."
echo "Rebasing branch $CURRBRANCH to latest master"
git fetch origin master && \
git rebase origin/master && \
echo "Checking out to master and pull" && \
git checkout master && \
git rebase origin/master && \
echo "Merging the change from $CURRBRANCH to master..." && \
git merge --ff-only $CURRBRANCH && \
git log | less 
echo "DONE. You may want to do one last test before pushing"

It’s worth mentioning that both GitHub and GitLab allows you to do the fast-forward (and squash) merge on it’s UI. But it’s configured on a per-repository basis, so if you don’t control the repository, you might have to ask your development team’s administrator to turn on the feature. You can read more about this feature in GitHub’s documentation and GitLab’s documentation. If you are interested in squashing the commits manually, but don’t know how, check out my previous post about squashing.

Minimal React.js Without A Build Step (Updated)

Thu, 08 Feb 2018 21:39:06 +0000

Back in 2016, I wrote a post about how to write a React.js page without a build step. If I remember correctly, at that time the official React.js site have very little information about running React.js without [Webpack][webpack], [in-browser Babel transpiler][babel] is not very stable and they are deprecating JSXTransformer.js. After the post my focus turned to browser backend projects and I haven’t touch React.js for a while. Now after 1.5 years, when I try to update one of [my React.js project][itinerary-viewer], I notice that the official site now has a clearer instruction on how to use React.js without a build step. So I’m going to write an update the post here.

You can find the example code on GitHub.

1. Load React.js from CDN instead of npm

You can use the official minimal HTML template here. The most crucial bit is the importing of scripts:

<script src="https://unpkg.com/react@16/umd/react.development.js"></script>
<script src="https://unpkg.com/react-dom@16/umd/react-dom.development.js"></script>
<script src="https://unpkg.com/babel-standalone@6.15.0/babel.min.js"></script>

If you want better error message, you might want to add the crossorigin attribute to the <script> tags, as suggested in the official document. Why the attribute you ask? As describe in MDN, this attribute will allow your page to log errors on CORS scripts loaded from the CDN.

If you are looking for better performance, load the *.production.min.js instead of *.development.js.

2. Get rid of JSX

I’m actually not that against JSX now, but If you don’t want to include the babel.min.js script, you can consider using the React.createElement function. Actually all JSX elements are syntatic sugar for calling React.createElement(). Here are some examples:

<h1>Hello Word</h1>

can be written as

React.createElement('h1', null, 'Hello World')

And if you want to pass attributes around, you can do

<div onClick={this.props.clickHandler} data={this.state.data}>
  Click Me!
</div>

React.createElement('div', {
                      'onClick': this.props.clickHandler, 
                      'data': this.state.data
                    }, 
                    'Click Me!')

Of course you can have nested elements:

<div>
  <h1>Hello World</h1>
  <a>Click Me!</a>
</div>

React.createElement('div', null, 
  React.createElement('h1', null, 'Hello World')
  React.createElement('a', null, 'Click Me!')
)

You can read how this works in the official documentation.

Split the React.js code into separate files

In the official HTML template, they show how to write script directly in HTML like:

<html>
  <body>
    <div id="root"></div>
    <script type="text/babel">

      ReactDOM.render(
        <h1>Hello, world!</h1>,
        document.getElementById('root')
      );

    </script>
  </body>
</html>

But for real-word projects we usually don’t want to throw everything into one big HTML file. So you can put everything between <script> and </script> in to a separate JavaScript file, let’s name it app.js and load it in the original HTML like so:

<html>
  <body>
    <div id="root"></div>
    <script src="app.js" type="text/babel"></script>
  </body>
</html>

The pitfall here is that you must keep the type="text/babel" attribute if you wants to use JSX. Otherwise the js script will fail when it first reaches a JSX tag, resulting an error like this:

SyntaxError: expected expression, got '<'[Learn More]        app.js:2:2

Using 3rd-party NPM components

Modules with browser support

You can find tons of ready-made React components on NPM, but the quality varies. Some of them are released with browser support, for example Reactstrap, which contains Bootstrap 4 components wrapped in React. In its documentation you can see a “CDN” section with a CDN link, which should just work by adding it to a script tag:

<!-- react-transition-group is required by reactstrap -->
<script src="https://unpkg.com/react-transition-group@2.2.1/dist/react-transition-group.min.js" charset="utf-8"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/reactstrap/4.8.0/reactstrap.min.js" charset="utf-8"></script>

then you can find the components in a gloabl variable Reactstrap:

<script type="text/babel" charset="utf-8">
  // "Import" the components from Reactstrap
  const {Button} = Reactstrap;

  // Render a Reactstrap Button element onto root
  ReactDOM.render(
    <Button color="danger">Hello, world!</Button>,
    document.getElementById('root')
  );
</script>

(In case you are curious, the first line is the destructing assignment of objects in JavaScript).

Of course it also works without JSX:

<script type="text/javascript" charset="utf-8">
  // "Import" the components from Reactstrap
  const {Button} = Reactstrap;

  // Render a Reactstrap Button element onto root
  ReactDOM.render(
    React.createElement(Button, {'color': 'danger'}, "Hello world!"),
    document.getElementById('root'),
  );
</script>

Modules without browser support

For modules without explicit browser support, you can still try to expose it to the browser with Browserify, as described in this post. Browserify is a tool that converts a Node.js module into something a browser can take. There are two tricks here:

Use the --standalone option so Browserify will expose the component under the window namespace, so you don’t need a module system to use it.
Use the browserify-global-shim plugin to strip all the usage of React and ReactDOM in the NPM module code, so it will use the React and ReactDOM we included using the <script> tags.

I’ll use a very simple React component on NPM, simple-react-modal, to illustrate this. First, we download this module to see what it looks like:

npm install simple-react-modal

If we go to node_modules/simple-react-modal, we can see a pre-built JavaScript package in the dist folder. Now we can install Browserify by npm install -g browserify. But we can’t just run it yet, because the code uses require('react') but we want to use our version loaded in the browser. So we need to install npm install browserify-global-shim and add the configuration to package.json:

// package.json
"browserify-global-shim": {
  "react": "React",
  "react-dom": "ReactDOM"
}

Now we can run

browserify node_modules/simple-react-modal \
  -o simple-react-modal-browser.js \
  --transform browserify-global-shim \
  --standalone Modal

We’ll get a simple-react-modal-browser.js file, which we can just load in the browser using the <script> tag. Then you can use the Modal like so:

<script type="text/javascript" charset="utf-8">
  // "Import" the components from Reactstrap
  const Modal = window.Modal.default;

  // Render a Reactstrap Button element onto root
  ReactDOM.render(
    React.createElement(Modal, 
      { 
        'show': true,
        'closeOnOuterClick': true 
      }, 
      React.createElement("h1", null, "Hello")
    ),
    document.getElementById('root')
  );
</script>

(There are some implementation detail about the simple-react-modal module in the above code, so don’t be worried if you don’t get everything.)

The benefits

Using this method, you can start prototyping by simply copying a HTML file. You don’t need to install Node.js, NPM and all the NPM modules that quickly make your small proof-of-concept page bloat.

Secondly, this method is compatible with the React-DevTools. Which is available in both Firefox and Chrome. So debugging is much easier.

Finally, It’s super easy to deploy the program. Simply drop the files into any web server (or use GitHub pages). The server doesn’t even need to run Node and NPM, any pure HTTP server will be sufficient. Other people can also easily download the HTML file and start hacking. This is a very nice way to rapidly prototype complex UIs without spending an extra hour setting up all the build steps (and maybe waste another 2 hour helping the team setting their environment).

Taking notes with MkDocs

Tue, 02 Jan 2018 20:14:58 +0000

I’ve been using TiddlyWiki for note-taking for a few years. I use them to keep track of my technical notes and checklists. TiddlyWiki is a brilliant piece of software. It is a single HTML file with a note-taking interface, where the notes you take are stored directly in the HTML file itself, so you can easily carry (copy) the file around and easily deploy it online for sharing. However, most modern browsers don’t allow web pages to access the filesystem, so in order to let TiddlyWiki save the notes, you need to rely on browser extensions or Dropbox integration service like TiddlyWiki in the Sky. But they still have some frictions.

So recently I started to look for other alternatives for note-taking. Here are some of the requirements I desire:

Notes are stored in non-proprietary format. In case the service/software is discontinued, I can easily migrate them to other tool.
Has some form of formatting, e.g. write in Markdown and render to HTML.
Auto-generated table of contents and search functionality.
Can be used offline and data is stored locally.
Can be easily shared with other people.
Can be version-controlled, and you won’t lose data if there is a conflict.

TiddlyWiki excels at 1 to 5, and I can easily sync it with Dropbox. However, I often forgot to click “save” in TiddlyWiki, and in some case when I accidentally create some conflict while syncing, Dropbox simply creates two copies and I have to manually merge them. There is also no version history so it’s hard to merge and look into the history.

Then I suddenly had an epiphany during shower: all I need is a bunch of Markdown files, version controlled by git, then I can use a static site generator to render them as HTML, with table of contents and client-side search. After some quick search I found MkDocs, which is a Python-based static-site generator for project documentations. It also have a live-reloading development server, which I really love.

Using MkDocs

MkDocs is really straight-forward to setup and use (under Linux). To install, simply use pip (assuming you have Python and pip installed):

sudo pip install mkdocs

Then you can create your document folder using

mkdocs new <project name>

The generated folder will have the following structure:

<project name>
├── docs
│   └── index.md
└── mkdocs.yml

You can then start to write documents in the docs folder. You can create subfolders to organize the Markdown files. To view the generated HTML, cd into the project folder, then run mkdocs serve, the development server will start to serve on port 8000. Opening 127.0.0.1:8000 in your browser then you can see the document. You can also run mkdocs build to generate the static HTML into the sites folder. The folder can then be hosted using any server.

The mkdocs.yml file contains some configuration. For example, you can use the ReadTheDocs theme by adding the line:

theme: readthedocs

If you wish to open the generated site locally using the file:// protocol, you can add this line:

use_directory_urls: false

When you have a note named foo.md, the generated file will be /foo/index.html. If use_directory_urls is set to true, the URL for linking to the foo.md page will be /foo/, which is a more modern URL naming convention. But for this routing to work, you must host the files in a web server. If you want to open it locally, you need to set the config to false and all the link will be /foo/index.html instead.

Migrating from TiddlyWiki

Moving all the notes from TiddlyWiki to MkDocs is very easy. If you are using TiddlyWiki 5.x+, you can go to “Tools” right under the search box, there is a “export all” button. Export the tiddlers (notes) to CSV. Then you can use the tiddly2md script to convert the CSV to individual .md files. If your tiddler has UTF-8 titles, you need to add a parameter encoding='utf-8' to the pd.read_csv() call in the script for it to work.

The exported Markdown files will lose the tag information, so if you are using tag as folder structure like me, you’ll have to manually create folders to arrange them. Some tiddlers using the old WikiText format will also be empty (I use Markdown in my TiddlyWiki, but there are some old ones from the old versions). You can use ls -lS to see which file has no content and manually fix them. After the .md files are in place, run mkdocs as usual.

Conclusion

MkDocs is a pretty simple but powerful tool for note-taking. You get all the benefit of editing the notes using plain text editor, and have them version controlled by git. But you also get a nicely rendered HTML version with search functionality. One thing I miss from TiddlyWiki is the ability to generate a single HTML file containing all the notes. (MkDocs generates a folder of separate HTML, CSS and JS files.) There are some scripts like mkdocs-combine that claims to do this using Pandoc, but I haven’t tested them.

Make LastPass Work Across App and Website

Mon, 16 Oct 2017 18:59:46 +0000

The Problem: My stock broker outsourced their Android app to a third-party company, so LastPass treat the desktop website and Android app as different sites. Although I can save them separately in LassPass, their password won’t synchronize with each other.

The Solution: LassPass do have a way to “merge” two sites into one, but the UI for doing so is not in the password vault page nor in the individual site’s configuration. Instead, the list of “equivalent” sites are under your account settings.

Here are the procedure to merge them:

Log-in to your LastPass desktop website. (I didn’t found the setting in the mobile app.)
Click your account icon on the top right and go to settings.
In your account settings, choose the “Equivalent Domains” tab.
Here you can add to the list of equivalent sites. For example my broker’s desktop site is using the domain “capital.com.tw”, but its app uses “mitake.com”, which is the outsource company that built the app. So I’ll add them into one entry so they are treated as equivalent.
Ta-dah! Now I can login to both the app and desktop site using the same LastPass entry.

Side note: Why I decided to use a password manager?

I avoid using a password manager for quite a while. I always believe it’s a bad idea to have a single point of failure. But after listening to a presentation by the security expert in my company, I decided to give it a try. The presenter mentioned that although LassPass had suffered from a few breaches in the past, the main password vault was never compromised, and they were very transparent about how and what had happened. So after I moved to Amsterdam and had to register tons of new online services, I simply can’t come up with a good algorithm to design my password that works across all sites. The risk of LastPass being completely hacked is relatively low comparing to my one password used across all sites being breached. If I use one password across all sites, then the level of security I get is equal to the security level of the weakest site I use (e.g. Yahoo?), so I would rather bet on LastPass.

Porting Chrome Extension to Firefox

Mon, 07 Aug 2017 06:37:22 +0000

Three years ago, I wrote the FocusBlocker to help me focus on my master thesis. It’s basically a website blocker that stops me from checking Facebook every five minute. But is different from other blockers like LeechBlock that requires you to set a fixed schedule. FocusBlocker lets you set a quota, e.g. I can browse 10 minutes of Facebook then block it for 50 minutes. So as long as you have remaining quota, you can check Facebook anytime. I’m glad that other people find it useful, and I even got my first donation through AMO because of happy users.

Since this extension serves my need, I’m not actively maintaining it or adding new features. But I was aware of Firefox’s transition from the legacy Add-on SDK to WebExtension API. So before WebExtension API is fully available, I started to migrate it to Chrome’s extension format. But I didn’t got the time to actually migrate it back to Firefox, until a user emails me asking for a WebExtension version. I looked into the statistics, the daily active user count drops from ~1000 to ~300. That’s when I rolled up my sleeve and actually migrated it in one day. (Although later I found out that the drop is not entirely due to users upgrading to newer Firefox, is because of this change.) Here is how I did it and what I’ve learned from the process.

What needs to be changed

To evaluate the scope of the work. We need to first look at what APIs I used. The FocusBlocker Chrome version uses the three main APIs:

chrome.tabs: to monitor new tabs opening and actually block existing tabs.
chrome.alarm: Set timers for blocking and unblocking.
chrome.storage.sync: To store the settings and persist the timer across browser restarts.

It’s nice that these APIs are all supported (at least the parts I used) in Firefox, so I don’t really need to modify any JavaScript code.

I loaded the manifest directly in Firefox’s about:debugging page (you can also consider use the convenient web-ext command line tool), but Firefox rejects it.

That’s because Firefox requires you to set a unique id for each extension(Not required if you use web-ext run. You can read more about the id requirement here), and you should (but not mandatory) also set a minimal version of Firefox on which the extension works, like so:

"applications": {
  "gecko": {
    "id": "focusblocker@shing.lyu",
    "strict_min_version": "48.0"
  }
},

There is one more modification need. In my Chrome extension I used the old options_page setting setting to set the preference page. But Firefox only support the newer options_ui. You can also apply browser’s system style for your settings page, so the UI looks like part of the Firefox setting. Firefox generalized the name from chrome_style to browser_style. So this is what I need to add to my manifest.json file (and remove the options_page setting):

"options_ui": {
  "page": "options.html",
  "browser_style": true
},

That’s all I need to port the extension from Chrome to Firefox. Super easy! The WebExtension team really did a good job on making the extensions compatible. In case you are curious, you can find the full source code of FocusBlocker on GitHub.

Publishing the extension on AMO

To publish the extension on addons.mozilla.org, you need to zip all the files in a zip and upload it. Here are some tips for passing the review more easily.

If you already have a legacy extension listed, upload the WebExtension-API-backed extension to replace it, so your user will have an easier time upgrading. (I chose to upload a separate listing, which is not recommended by the Add-on team.)
Don’t pack any unnecessary file into the zip, exclude all the temporary test files from the zip.
Remove or comment out all the console.log() calls. Although it’s not a strict requirement, but it will make the review process much smoother.
If you use any third party library, consider including (i.e. “vendoring”) the file into the zip, or at least upload the source for review.
If you’ve upload one version and you’d like to make some modifications or fix, you need to bump the version number, no matter how small the change is.

Firefox is planning to completely roll out the new format in version 57 (around November, 2017). So if you have a legacy Firefox extension, or a Chrome extension you want to convert, now is a perfect timing.

If you want to try out the new FocusBlocker, please head to the install page. You can also find the Chrome version here.

Edit (2017/8/30): Andreas from the Mozilla Add-on team points out a few errors, I’ve modified the original text to reflect that:

Do NOT create a new version of the extension on AMO, upload and replace your legacy extension using the same listing.
The user drop is related to https://blog.mozilla.org/addons/2017/06/21/upcoming-changes-usage-statistics/
The web-ext run should work without an ID
strict_min_version is not mandatory

Install Ubuntu 16.04 on ThinkPad 13 (2nd Gen)

Tue, 11 Jul 2017 07:48:30 +0000

It has been a while since my last post. I’ve been busy for the first half of this year but now I got more free time. Hopefully I can get back to my usual pace of one post per months.

My old laptop (Inhon Carbonbook, discontinued) had a swollen battery. I kept using it for a few months but then the battery squeezed my keyboard so I can no longer type correctly. After some research I decided to buy the ThinkPad 13 model because it provides descent hardware for its price, and the weight (~1.5 kg) is acceptable. Every time I got a new computer the first thing is to get Linux up and running. So here are my notes on how to install Ubuntu Linux on it.

TL;DR: Everything works out of the box. Just remember to turn off secure boot and shrink the disk in Windows before you install.

Some assumptions

I’m using ThinkPad 13 2nd Gen bought in June 2017
I’m using Ubuntu Linux 16.04.2 LTS
I use Linux as my primary operating system, but still keeps Windows for rare occasions.

Before Installing Ubuntu

First we need to clean up some disk space for Ubuntu. But if you are going to delete Windows completely you can skip the following steps.

(In Windows) Right click on the start menu, select “PowerShell (administrator)”, then run diskmgmt.msc.
Right click the C: disk and select the shrink disk option.

Before we can install Ubuntu, we need to disable the secure boot feature in BIOS.

Press Shift+restart to be able to get into BIOS, otherwise you’ll keep booting into Windows.
Press Enter followed by F1 to go into BIOS during boot.
Disable Secure Boot.

UEFI boot seems to work with Ubuntu 16.04’s installer, so you can keep all the UEFI options enabled in the BIOS. Download the Ubuntu installation disk and use a bootable USD disk creator that supports UEFI, for example Rufus.

Installing Ubuntu

Installing Ubuntu should be pretty straightforward if you’ve done it before.

Go to BIOS again and set the USB drive as top priority boot medium.
Boot into Ubuntu, select “Install Ubuntu”.
Follow the installer wizard.
Select “Something else” when asked how to partition the disk.
Create a linux-swap partition of 4GB for 8GB of RAM. I followed Redhat’s suggestion, but there are different theories out there, so use your own judgement here.
Create a EXT4 main disk and set the mount point to /
After the installer finished, reboot. You should see the GRUB2 menu. The Windows options should also work without trouble.

What works?

Almost everything works out of the box. All the media keys, like volume control, brightness and WiFi and Bluetooth toggle is recognized by the built-in Unity desktop. But I am using i3 so I have to set them up myself, more on this later. The USB Type-C port is a nice surprise for me. It supports charging, so you can charge with any Type-C charger. I tested Apple’s Macbook charger and it works well. I also tested Apple’s and Dell’s Type-C multi-port adapter and both works, so I can plug my external monitor and my keyboard/mouse to it so it works like a docking station.

A side note, I’m also glad to find that Ubuntu now use the Fcitx IME by default. Most of the IME bugs I found in previous versions and ibus are now gone.

What doesn’t work?

Not that I’m aware of. The only complaint I have is that the Ethernet and Wi-Fi naming method has changed somehow (e.g. enp0s31f6 instead of eth0). But I believe that is something we can fix by software. People also complain that the power button and the hinge is not very sturdy. But I guess that’s the compromise you have to make for the relatively cheap price.

More on setting up media keys for i3 window manager

Since I use the i3 window manager, I don’t have Unity to handling all my media keys’ functionality. But it’s not hard to set them up using the following i3 config:

bindsym XF86AudioRaiseVolume exec amixer -q set Master 2dB+ unmute
bindsym XF86AudioLowerVolume exec amixer -q set Master 2dB- unmute
bindsym XF86AudioMute        exec amixer -D pulse set Master toggle # Must assign device "pulse" to successfully unmute

# Only two level of brightness
bindsym XF86MonBrightnessUp    exec xrandr --ouptut eDP-1 --brightness 0.8
bindsym XF86MonBrightnessDown  exec xrandr --ouptut eDP-1 --brightness 0.5

The only drawback is that the LED indicator on the mute buttons mighty be out of sync with the real software state.

Conclusion

Overall, ThinkPad 13 is a descent laptop for its price range. Ubuntu 16.04 works out of the box. No sweat! If you are looking for a good Linux laptop, but can’t afford ThinkPad X1 Carbon or Dell’s XPS 13/15, ThinkPad 13 might be a good choice.

An Overview of Asia Tech Conferences in 2017

Sat, 21 Jan 2017 08:22:34 +0000

I’ve been attending and even talking at tech conferences for some time. One of the challenge is to keep track of when those conference will take place. Also there is no single list of all conferences I’m interested. There are some website that collects them, but they often missed some community-organized events in Asia. Or there are some community-maintained list of open source conferences (Thanks Barney!), but they don’t include for-profit conferences.

Therefore I build a simple website that collects all conferences I know in Asia, focusing on open source software, web, and startup:

https://asia-confs.github.io/asia-tech-confs/

#The Technology Stack Since I don’t really need dynamic-generated content, I use the Jekyll static site generator. For the look and feel, I use the Material Design Lite (MDL) CSS framework. (I did try other material design frameworks like Materialize or MUI, but MDL is the most mature and clean one I can find.)

One of the challenge is to provide the list in different languages. I found a plugin-free way to make Jekyll support I18N (Internationalization). The essence is to create language specific sub-directories like en/index.md and zh_tw/index.md. Then put all language specific string in the index.md files. One pitfall is that by adding another level of directory, the relative paths (e.g. path to CSS and JS files) might not work, so you might need to use absolute path instead. For Traditional and Simplified Chinese translation, I’m too lazy to maintain two copy of the data. So I use a JavaScript snippet to do the translation on-the-fly.

How to Contribute

If you know any conference, meetup or event that should be on the list, please feel free to drop and email to asia.confs@gmail.com. Or you can create a pull request or file and issue to our GitHub repo.

Enjoy the conferences and Happy Chinese New Year!

Vim QuickFix for Rust and Servo

Sun, 25 Dec 2016 13:00:00 +0000

While working on a compiled language like Rust, a typical workflow is compile -> find the errors in the compiler message -> find the file containing the error -> edit -> re-compile. But usually there are lots of errors scattered around the compiler log, and to identify the filename and line number, and manually open the file to the correct line in an editor is a tedious job. Vim’s quickfix streamline the process by collecting the errors into the split panel in vim, and allow you to navigate through the errors using the :cnext (next error) and :cprev (previous command). While you navigate to an error, the corresponding source file will be opened in the main vim window and jump directly to the line where the error is.

When you run :make inside vim, vim will execute the external compile tool (default: Makefile’s make’), then the compiler output will be parsed to extract the file name, line number, etc. By default, vim understands Makefile-based projects with C/C++ compiler output. But you can change the behavior by setting the makeprg (defines what :make does) and the errorformat (error message parsing pattern). (You can run :help options and read “22 running make and jumping to errors” for detail.)

rust.vim

The good news is: you don’t need to write the makeprg and errorformat yourself. The rust.vim vim plugin will do that for you. Installation is pretty simple, based on which vim plugin manager you use, you’ll find the instruction on the project’s GitHub README. I use Vundle, so I add the following line to my ~/.vimrc:

Plugin 'rust-lang/rust.vim'

Then run :PluginInstall in vim.

Now for ordinary Rust project which uses cargo, you can set the compiler as cargo by :compiler cargo. Then if you run :make build, vim will execute cargo build for you.

You can also pass other parameters like you’d to cargo, e.g. :make test will run cargo test.

After the compilation finished, you will see the source code being loaded, but you don’t see the error message. That’s because we haven’t open the quickfix panel yet. Now type :copen then you’ll see a list of errors like this:

The vim main window will load the source file where the first error is pointing to. After you finish fixing it, you can use :cnext command to jump to the next one.

Servo

This all works very well for cargo-based Rust projects, but what if you are working on Servo, which uses mach as the build tool? The solution is simple, you still run :compiler cargo to get the errorformat setup. But then you run :set makeprg=./mach to make :make run ./mach instead of cargo.

If you want to avoid typing the commands every time you restart vim, you can set them in your ~/.vimrc, but if you work on Servo and other cargo-based Rust projects at the same time, you can actually set a per-directory based configuration. For example, you can put a .vimrc (.nvimrc if you use NeoVim) in your Servo directory like so:

compiler cargo
set makeprg=./mach

autocmd QuickFixCmdPost [^l]* nested cwindow
autocmd QuickFixCmdPost    l* nested lwindow

The last two lines makes quickfix window show up automatically right after the compilation. You can omit them if you want.

Vim will not load the local .vimrc file by default to avoid running malicious .vimrcs. You need to enable them in your global ~/.vimrc file by adding these lines:

set exrc 
set secure

That’s it! Now you can have different compiler setup in different projects.

Beginner's guide to git rebasing and squashing

Tue, 08 Nov 2016 06:29:17 +0000

I wrote this post on the Servo wiki to help beginners getting started with rebasing and squashing, two of the most terrifying operations you’ll face if you are not familiar with git. I’m cross posting this here for people working on other projects.

Big thanks to Wafflespeanut who proofread the post, any error you found here is my own.

Suppose you’ve created a pull request following the checklist, but the reviewer asks you to fix something, do a rebase or squash your commits, how exactly do you do that? If you have some experience with git, you might want to check the GitHub workflow for a quick overview. But if you are not familiar with git enough, we’ll teach you how to do these common operations in detail.

Fixing review comments

Once you reviewer reviewed your patch, he/she might leave some comments asking you to fix something. So you edit the source code, then you will probably do something like this.

git add <the file you fixed> then git commit, write a commit message telling people what you’ve fixed. (You might also check out the --fixup option for git commit in the workflow doc.)
Simply git push to the same remote branch which you’ve created the PR with. The GitHub pull request page will pick up your changes, and hide those review comments you’ve fixed.

If your fix is trivial, and you have a single commit ready for merge, then you can consider using git commit --amend to add the change directly to your last commit. Then, all you need to do is git push -f to force push to the branch at your fork.

Rebasing

Sometimes, if someone merged new code while your patch is still in review, git might not be able to figure out how to apply your patch on top of the new code. In this case, our bors-servo bot will notify you with a helpful message:

☔️ The latest upstream changes (presumably #12345) made this pull request unmergeable. Please resolve the merge conflicts.

… and the GitHub UI will say “This branch has conflicts that must be resolved”. This is when you need a rebase. Here, we’ll explain the power of rebasing with a simple example.

Suppose the servo/servo tree is like this before you start working on a bug, in which R is the latest commit:

P -> Q -> R (<= remote servo/servo)

You create a new branch from R, then you add your own fixes X and Y:

P -> Q -> R   (<= remote servo/servo)
           \
            .--> X -> Y (<= your local branch)

But if someone merged their PR S before you, and he/she modified the code which you had also been working on, then git might fail to know how to merge the changes from both of you. So, we should fix this by ourselves.

P -> Q -> R ---> S
           \
            .--> X -> Y

But you cannot do that with a git pull, because it will create a merge commit, which will mess up the git history. A git pull will make the tree look like this, in which the M commit contains stuff from your X, Y and S

P -> Q -> R ---> S -------.
           \               \
            \               v
             .--> X -> Y -> M

We want to fix this by rebasing, which means re-attach our changes X and Y to the new root S, like so:

P -> Q -> R -> S
                \
                 .--> X -> Y

Here is how we do it:

Let’s assume your servo repo is cloned from your own fork, (if you run git remote -v, you can see the origin points to something like git@github.com:<your username>/servo.git rather than git@github.com:servo/servo.git).
You need to create a new remote called ‘upstream’ that points to the servo/servo branch, so we can download the latest code. Run git remote add upstream git@github.com:servo/servo.git
Now let’s download the latest code from servo/servo, but don’t try to merge them: git fetch upstream.
Now we can rebase our X and Y on top of the latest change, run git rebase upstream/master.
Git might ask you to fix conflicts. We’ll get into that now.

Fixing conflicts

First run git status, it will tell you which file was both modified. Open those files one by one in them, you’ll see lines like this:

<<<<<<<<<<< HEAD
use std::cmp::{max, min};
===========
use std::cmp::{max, PartialEq};
>>>>>>>>>>> Your Commit

This means that in the commit S, the author wants to add min to the use line, but in your commit you want to add PartialEq. (Lines between the <<< and === are the version on servo/servo; the lines between === and >>> are your local version.) A way to fix this is to include both, so you can delete all the lines from the >>> to <<<, and replace them with the correct code:

use std::cmp::{max, min, ParitalEq};

After you fixed all the conflicts, you can run git add <the files you edited>, then git rebase --continue. You might need to repeat this action multiple times until every conflict is resolved. (In case you messed up, you can always run git rebase --abort to start over).

Squashing

Once the reviewer approves your PR, he/she might ask you to “squash” the commits. There are a lot of reasons for this. If you have a lot of fixup commits, and you merge all of them directly into servo/servo, the git history will be bloated (which is something we don’t want). Or, if your recent commit fixes your previous commit in the same PR, then you could’ve simply rebased it (we prefer fixing the mistakes made by you).

Anyway, using the last example, if your change consists of two commits X and Y, we want to squash them into a single commit Z.

P -> Q -> R
           \
            .--> X -> Y

P -> Q -> R
           \
            .--> Z

To achieve this, we can use the git rebase -i command.

First we need to identify the last commit before the ones you want to merge, which is R in our example. Run git log and remember the hash of R.

Run git rebase -i <hash of R>, this will bring up your default text editor with a content like:

 pick 7de252c X
 pick 02e5bd1 Y

 # Rebase 170afb6..02e5bd1 onto 170afb6 (2 command(s))
 #
 # Commands:
 # p, pick = use commit
 # r, reword = use commit, but edit the commit message
 # e, edit = use commit, but stop for amending
 # s, squash = use commit, but meld into previous commit
 # f, fixup = like "squash", but discard this commit's log message
 # x, exec = run command (the rest of the line) using shell
 # d, drop = remove commit
 #
 # These lines can be re-ordered; they are executed from top to bottom.
 #
 # If you remove a line here THAT COMMIT WILL BE LOST.
 #
 # However, if you remove everything, the rebase will be aborted.
 #
 # Note that empty commits are commented out

Keep the first commit as pick, and change all the other pick to squash (or s for short):

 pick 7de252c X
 squash 02e5bd1 Y

 # Rebase 170afb6..02e5bd1 onto 170afb6 (2 command(s))
 ...

Now save and quit the text editor, the rebase will run until the end. You might meet conflicts like you do in rebasing. Fix then using the same method described in the previous section.
After the rebase is finished, the editor will pop-up again, now you can write the commit message for the new commit Z.
git push -f to push the squashed commit to GitHub (and update the PR).

If you made any mistake right after you run step 2, you can abort by deleting every line in the text editor then save and exit. If you mess up fixing the conflicts, you can also run git rebase --abort to reset everything and start over.

Ask for help

Working on git for a personal project is very different from collaborating on giant open-source projects like Servo. So, if this guide doesn’t solve your problem, feel free to ask your reviewer in the pull request or ask people on IRC (#servo on irc.mozilla.org).

Mutation Testing in JavaScript Using Stryker

Tue, 11 Oct 2016 01:00:00 +0000

Earlier this year, I wrote a blog post introducing Mutation Testing in JavaScript using the Grunt Mutation Testing framework. But as their NPM README said,

We will be working on (gradually) migrating the majority of the code base to the Stryker.

So I’ll update my post to use the latest Stryker framework. The following will be the updated post with all the code example migrated to the Stryker framework:

Code example: github link

Last November (2015) I attended the EuroStar Software Testing Conference, and was introduced to a interesting idea called mutation testing. Ask yourself: “How do I ensure my (automated) unit test suite is good enough?”. Did you miss any important test? Is your test always passing so it didn’t catch anything? Is there anything un-testable in your code such that your test suite can never catch it?

Introduction to Mutation Testing

Mutation testing tries to “test your tests” by deliberately inject faults (called “mutants”) into your program under test. If we re-run the tests on the crippled program, our test suite should catch it (i.e. some test should fail.) If you missed some test, the error might slip through and the test will pass. Borrowing terms from Genetics, if the test fails, we say the mutant is “killed” by the test suite; on the opposite, if the tests passes, we say the mutant survived.

The goal of mutation testing is to kill all mutants by enhancing the test suite. Although 100% kill is usually impossible for even small programs, any progress on increasing the number can still benefit your test a lot.

The concept of mutation testing has been around for quite a while, but it didn’t get very popular because of the following reasons: first, it is slow. The number of possible mutations are just too much, re-compiling (e.g. C++) and re-run the test will take too long. Various methods has been proposed to lower the number of tests we need to run without sacrifice the chance of finding problems. The second is the “equivalent mutation” problem, which we’ll discuss in more detail in the examples.

Mutation Testing from JavaScript

There are many existing mutation testing framework. But most of them are for languages like Java, C++ or C#. Since my job is mainly about JavaScript (both in browser and Node.js), I wanted to run mutation testing in JavaScript.

I have found a few mutation testing frameworks for JavaScript, but they are either non-open-source or very academic. The most mature one I can find so far is the Stryker framework, which is released under Apache 2.0 license.

This framework supports mutants like changing the math operators, changing logic operators or even removing conditionals and block statements. You can find a full list of mutations with examples here.

Setting up the environment

You can follow the quickstart guide to install everything you need. The guide has a nice interactive menu so you can choose your favorite build system, test runner, test framework and reporting format. In this blog post I’ll demonstrate with my favorite combination: Vanilla NPM + Mocha + Mocha + clear-text.

You’ll need node and npm installed. (I recommended using nvm).

There are a few Node packages you need to install,

sudo npm install -g --save-dev mocha
npm install --save-dev stryker styker-api stryker-mocha-runner

Here are the list of packaged that I’ve installed:

"devDependencies": {
	"mocha": "^2.5.3",
	"stryker": "^0.4.3",
	"stryker-api": "^0.2.0",
	"stryker-mocha-runner": "^0.1.0"
}

A simple test suite

I created a simple program in src/calculator.js, which has two functions:

// ===== calculator.js =====
function substractPositive(num1, num2){
  if (num1 > 0){
    return num1 - num2;
  }
  else {
    return 0
  }
}

function add(num1, num2){
  if (num1 == num2){
    return num1 + num2;
  }
  else if (num1  num2){
    return num1 + num2;
  }
  else {
    return num1 + num2;
  }
}

module.exports.substractPositive = substractPositive
module.exports.add = add;

The first function is called substractPositive, it substract num2 from num1 if num1 is a positive number. If num1 is not positive, it will return 0 instead. It doesn’t make much sense, but it’s just for demonstrative purpose.

The second is a simple add function that adds two numbers. It has a unnecessary if...else... statement, which is also used to demonstrate the power of mutation testing.

The two functions are tested using test/test_calculator.js:

var assert = require("assert");
var cal = require("../src/calculator.js")

describe("Calculator", function(){
  it("substractPositive", function(){
    assert.equal("2", cal.substractPositive(1, -1));
  });

  it("add", function(){
    assert.equal("2", cal.add(1, 1));
  });
})

This is a test file running using mocha, The first verifies substractPositive(1, -1) returns 2. The second tests add(1,1) produces 2. If you run mocha in your commandline, you’ll see both the test passes. If you run mocha in the commandline you’ll see the output:

% mocha

  Calculator
    ✓ substractPositive
    ✓ add

  2 passing (6ms)

So this test suite looks OK, it exercises both function and verifies its output, but is it good enough? Let’s verify this by some mutation testing.

Running the mutation testing

To run the mutation testing, we first need to create a config file for the stryker mutator. Create a file called stryker.conf.js in the project’s root directory and paste the following input into it:

// stryker.conf.js
module.exports = function(config){
  config.set({
    files: [
        // Add your files here, this is just an example:
        { pattern: 'src/**/*.js', mutated: true, included: false},
        'test/**/*.js'
    ],
    testRunner: 'mocha',
    testFramework: 'mocha',
    testSelector: null,
    reporter: ['clear-text', 'progress']
  });
}

You can see we choose mocha as the testRunner and testFramework, and we tell the Stryker framework to run the test files in test/**/*.js, and the source files are in src/**/*.js.

Then we need to tell npm how to run Stryker in the package.json file. Simply add the following code in your package.json configuration:

"scripts": {
  "stryker": "stryker -c stryker.conf.js"
},

This will add a npm run stryker command to npm run and execute the stryker -c stryker.conf.js script.

Finding Missing Tests

Now if we run npm run stryker, you’ll see the following test result:

% npm run stryker

> @ stryker /home/shinglyu/workspace/mutation-testing-demo/stryker
> stryker -c stryker.conf.js

[2016-10-07 15:14:51.997] [INFO] InputFileResolver - Found 1 file(s) to be mutated.
[2016-10-07 15:14:52.129] [INFO] Stryker - Initial test run succeeded. Ran 2 tests.
[2016-10-07 15:14:52.151] [INFO] Stryker - 22 Mutant(s) generated
[2016-10-07 15:14:52.153] [INFO] TestRunnerOrchestrator - Creating 8 test runners (based on cpu count)
SSSSSSSSSSSSS..S......
Mutant survived!
/home/shinglyu/workspace/mutation-testing-demo/stryker/src/calculator.js: line 17:7
Mutator: BlockStatement
-     else {
-       return num1 + num2;
-     }
+     else {
+   }

Tests ran:
    Calculator substractPositive
    Calculator add

Mutant survived!
/home/shinglyu/workspace/mutation-testing-demo/stryker/src/calculator.js: line 18:11
Mutator: Math
-       return num1 + num2;
+       return num1 - num2;

Tests ran:
    Calculator substractPositive
    Calculator add

Mutant survived!
/home/shinglyu/workspace/mutation-testing-demo/stryker/src/calculator.js: line 14:11
Mutator: RemoveConditionals
-     else if (num1 > num2){
+     else if (false){

Tests ran:
    Calculator substractPositive
    Calculator add

Mutant survived!
/home/shinglyu/workspace/mutation-testing-demo/stryker/src/calculator.js: line 14:23
Mutator: BlockStatement
-     else if (num1 > num2){
-       return num1 + num2;
-     }
+     else if (num1 > num2){
+   }

Tests ran:
    Calculator substractPositive
    Calculator add

//(omitted for brevity)

22 mutants tested.
0 mutants untested.
0 mutants timed out.
8 mutants killed.
Mutation score based on covered code: 36.36%
Mutation score based on all code: 36.36%
[2016-10-07 15:14:52.468] [INFO] fileUtils - Cleaning stryker temp folder /home/shinglyu/workspace/mutation-testing-demo/stryker/.stryker-tmp

As you can see, it tested 22 kinds of mutations, but 14 (22-8=14) of them survived!

Let’s look at a survived mutant:

Mutant survived!
/home/shinglyu/workspace/mutation-testing-demo/stryker/src/calculator.js: line 2:6
Mutator: ConditionalBoundary
-     if (num1 > 0){
+     if (num1 >= 0){

It tells us that by replacing the > with >= in line 2 of our calculator.js file, the test will pass.

If you look the line, the problem is pretty clear

// ===== calculator.js =====
function substractPositive(num1, num2){
  if (num1 > 0){ // <== This line
    return num1 - num2;
  }
  else {
    return 0
  }
}

We didn’t test the boundary values, if num1 == 0, then the program should go to the else branch and returns 0. By changing the > to >=, the program will go into the return num1 - num2 branch instead and returns 0 - num2!

This is one of the problem mutation testing can solve, it tells you which test case you missed. The solution is very simple, we can add a test like this:

it('substractPositive', function(){
  assert.equal('0', cal.substractPositive(0, -1));
});

If you run mutation testing again, the problem with the substractPositive function should go away.

Equivalent Mutation and Dead Code

Sometimes a mutation will not change the behavior of the program, so no matter what test you write, you can never make it fail. For example, a mutation may disable caching in your program, the program will run slower but the behavior will be exactly the same, so you’ll have a mutation you can never kill. This kind of mutation is called “equivalent mutation”.

Equivalent mutation will make you overestimate your mutation survival rate. And they time to debug, but may not reveal useful information about your test suite. However, some equivalent mutations do reveal issues about your program under test.

Let look at the mutation results again:

Mutant survived!
/home/shinglyu/workspace/mutation-testing-demo/stryker/src/calculator.js: line 11:6
Mutator: ReverseConditional
-     if (num1 == num2){
+     if (num1 != num2){

Mutant survived!
/home/shinglyu/workspace/mutation-testing-demo/stryker/src/calculator.js: line 14:11
Mutator: RemoveConditionals
-     else if (num1 > num2){
+     else if (false){

Mutant survived!
/home/shinglyu/workspace/mutation-testing-demo/stryker/src/calculator.js: line 14:23
Mutator: BlockStatement
-     else if (num1 > num2){
-       return num1 + num2;
-     }
+     else if (num1 > num2){
+   }

Mutant survived!
/home/shinglyu/workspace/mutation-testing-demo/stryker/src/calculator.js: line 15:11
Mutator: Math
-       return num1 + num2;
+       return num1 - num2;

If you look at the code, you’ll find that all the branches of the if...else... statement returns the same thing. So no matter how you mutate the if...else... conditions, or even remove one branch that was not reached, the function will always return the correct result.

10 function add(num1, num2){
11   if (num1 == num2){
12     return num1 + num2;
13   }
14  else if (num1 > num2){
15     return num1 + num2;
16   }
17  else {
18     return num1 + num2;
19   }
20 }

Because we found that the if...else... is useless, we can simplify the function to only three lines:

function add(num1, num2){
  return num1 + num2;
}

If you run the mutation test again, you can see all the mutations being killed.

This is one of the side benefit of equivalent mutations, although your test suite is fine, it tells you that your program code has dead code or untestable code.

Next Steps

By now you should have a rough idea about how mutation testing works, and how to actually apply them in your JavaScript project. If you are interested in mutation testing, there are more interesting questions you can dive into, for example, how to use code coverage data to reduce the test you need to run? How to avoid equivalent mutations? I hope you’ll find many interesting methods you can apply to your testing work. You can submit issues and suggestions to the Stryker GitHub repository, or even contribute code to them. The team is very responsive and friendly. Happy Mutation Testing!

2016 COSCUP, Modern Web and Taiwan Code Sprint

Sun, 04 Sep 2016 12:40:00 +0000

August was quite a busy month for me. I went to two conferences and one workshop to promote Servo and Rust. Here are what I’ve observed in those events.

COSCUP

COSCUP is one of the largest volunteer-organized conference on open source software in Taiwan. When I was a student I tried to attend all the sessions, wishing to learn from the talks. But in recent years I realize that it’s really not about the talks, but more about meeting people. Half of the talks are about introducing people to new tools or libraries. You can learn them by yourself more efficiently by reading tutorial articles online. The most rewarding part is actually socializing with people. Because most of the open source participants will attended this conference, you can easily meet a lot of new friends through mutual friends. So this year I spend most of my time at social events, for example the speaker dinner party, Mozilla BoF (bird of a feather), and chatting with my friend’s friends.

But I still listened to quite a few interesting sessions. This year’s talk (schedule) has a high percentage of Internet of Things talk. But they are still fundamental programming stuff like Linux kernel and LLVM. There are also a lot of presence of Mozilla-related topics: connected devices projects like Webby End SensorWeb, Bob Chao’s talk on Mozilla Community Space Project, and Gary Kwong talked about fuzzy testing in the Unconf. We also have a booth for Mozilla, in which we show off the Taiwan community. I also attended the BoF and had a nice chat with the community. My talk also went pretty well, I’ll talk more about them later.

Overall it was a nice event. I meant a lot of old and new friends, including two or three people approaching me saying that they want to contribute to Servo. The atmosphere is pretty relaxed, it feels more like an open source community reunion than a technical conference. One of the best idea I saw in the conference is the souvenir sticker collection book. This solves a huge problem we tech geeks have: having too many stickers but too little laptop surface area.

Modern Web

The second conference I attended is the Modern Web Conference hosted by iThome, a technical publisher in Taiwan. The conference is much more expensive (~$200 USD) so the logistics are better. Each room has a dedicated AV team which handles the projector, audio and video recording. A good arrangement is that after a talk, the speaker can move to a Q&A space outside the meeting room, so people can ask follow-up questions without blocking the next session. They also have a cool Dojo booth, you can get a 15 minute hands-On tutorial on topics like frontend testing and responsive CSS design, etc.

The hottest topics in the conference is React.js and React Native. A lot of companies are trying to merge their frontend and Android/iOS app team, to encourage code sharing and create a unified experience across platforms. To my surprise, most them are pretty satisfied with the solution, I believe that’s partially because for most of them, the app is not the most critical factor in their business. They just want to have some presence in every platform, so having one or two taking care of all the platform sound like a better investment then having dedicated engineer for each platform. There is even a thing called “React Native for Web”, we are porting web to native, now we are porting them back to web?! There is also a high interest in backend performance tuning. There are talks about NGINX performance tuning, container orchestration, and various tricks for speeding up your website from even the network protocol level. I also heard an interesting talk about the status of HTML emails. Although I know HTML email is hard to work with, I didn’t know the support is so different across mail clients.

The Servo Talks

I gave roughly the same talk in the above two conferences. The COSCUP audiences are more interested in the system library side of Servo, they asked about do we have Wayland support (yes, we do), and the status of our media support through ffmpeg. The Modern Web crowd are less interested in our internal implementation, but on how and when Servo will become a real product. They also asked a lot about the future of Servo together with Firefox, which I think might get clearer in the following months.

WebRender’s performance and its promise to allow web dev to write 60 fps animation without understanding the internal implementation resonates well with the audiences. There was a talk in Modern Web about how to get the best CSS animation performance, which is basically about which API for avoid, hopefully with WebRender we don’t need to worry about them anymore. They also wants to know the detail about out Web Platform Test effort. In general, people excited rather then worried (by the need to support yet another browser) about Servo. The doge sticker also helped me attracting the crowd! The Modern Web staff also helped me making the sticker into this cute button:

Taiwan Code Sprint

This is the third event I attended in one week, and also the most exhausting one. This is a new event hosted by a bunch of collage students, who wants to connect potential open source project contributors with mentors. I signed up as a Servo mentor and received roughly six applications. They rented two classrooms in the National Normal University in Taipei, and all mentors and students stay there for 2 days to hack on easy issues. There are people from Firefox, Ubuntu, LibreOffice, Linux Kernel, etc.

Thanks to the help from the Servo community around the world, especially jdm and KiChjang, we were able to get around 14 patches landed in just two days. The new contributors are very impressed by the friendliness of the contribution flow. What confuses them is some Rust-specific syntax like matching on Rust’s enum, or working with unwrap(). Some of the contributors who are not familiar with the GitHub workflow needs some help with rebasing, pushing new modifications and squashing, but in general they are able to work through the easy bugs pretty fast.

With six of the new contributors joining the Servo project, we now have about fifteen Servo contributors in Taiwan. Our Rust book study group also finished reading the basic part of the book, and we are moving on to build side projects in Rust. Hopefully we can create a strong Servo/Rust community here in Taiwan!

Identify Performance Regression in Servo

Mon, 18 Jul 2016 02:00:00 +0000

Performance has always been a key focus for the Servo browser engine project. But just measure the performance through profilers and benchmarks is not enough. The first impression to a real user is the page load time. Although many internal, non-visible optimizations are important, we still want to make sure our page load time is doing well.

Back in April, I opened this bug #10452 to start planning the page load test. With the kind advice from the Servo community and the Treeherder people, we finally settled for a test design similar to the Talos test suite, and decided to use Perfherder for visualization.

Test Design

Talos is a performance test suite designed for Gecko, the browser engine for Firefox. It has many different kinds of tests, covering user-level UI testing and benchmarking. But what we really care about is the TP5 page load test suite. As the wiki says, TP5 use Firefox to load 51 scrapped websites selected from the Alexa Top 500 sites of its time. Those sites are hand-picked, then downloaded and cleaned to remove all external web resources. Then these web pages are hosted on a local server to reduce network latency impact.

Each page is tested three times for Servo, then we take the medium of the three. (We should test more times, but it will take too long.) Then all the mediums are averaged using geometric mean. Geometric mean has a great property that even if two test results are of different scale (e.g. 500 ms v.s. 10000 ms), if any one of them changed by 10%, they will have equal impact on the average.

Visualization

Talos test results for Gecko have been using Treeherder and Perfherder for a while. The former is a dashboard for test results per commit; the latter is a line plot visualization for the Talos results. With the help from the Treeherder team, we were able to push Servo performance test results to the Perfherder dashboard. I had a blog post on how to do this. You’ll see screenshots for Treeherder and Perfherder in the following sections.

Implementation

We created a python test runner to execute the test. To minimize the effect of hardware differences, we run the Vagrant (VirtualBox backend) virtual machine used in Servo’s CI infrastructure. (You can find the Vagrantfile here). The test is scheduled by buildbot and runs every midnight.

The test results are collected into a JSON file, then consumed by the test result uploader script. The uploader script will format the test result, calculate the average and push the data to Treeherder/Perfherder throught the Python client

The 25% Speedup!

A week before the Mozilla London Workweek, we found a big gap in the Perfherder graph. The average page load time changed from about 2000 ms to 1500 ms on June 10th.

We were very excited about the significant improvement. Perfherder conveniently links to the commits in that build, but there are 26 commits in between.

You should notice that there are many commits by the “bors-servo” bot, who is out automatic CI bot that does the pre-commit testing and auto-merging. Those commits are the merge commits generated when the pull request is merged. Other commits are commits from the contributors branch, so they may appear earlier then the corresponding merge commit. Since we only care when the commit gets merged to the master branch, not when the contributor commits to their own branch, we’ll only bisect the merge commits by bors-servo.

Buildbot provides a convenient web interface for forcing a build on certain commits.

You can simply type the commit has in the “Revision” field and buildbot will checkout that commit, build it and run all the tests.

You can track the progress on the Buildbot waterfall dashboard.

Finally, you’ll be able to see the test result on Treeherder and Perfherder.

The performance improvement turns out to be the result of this patch by Florian Duraffourg, he use a hashmap to replace a slow list search.

Looking Forward

In the near future, we’ll focus on improving the framework’s stability to support Servo’s performance optimization endeavor. We’ll also work closely with the Treeherder team to expand the flexibility of Treeherder and Perfherder to support more performance frameworks.

If you are interested in the framework, you can find open bugs here, or join the discussion in the tracking bug.

Thanks William Lachance for his help on the Treeherder and Perfherder stuff, and helped me a lot in setting it up on Treeherder staging server. And thanks Lars Bergstrom and Jack Moffit for their advice throughout the planning process. And thanks Adrian Utrilla for contributing many good features to this project.

Show Firefox Bookmark Toolbar in Fullscreen Mode

Mon, 20 Jun 2016 11:05:00 +0000

By default, the bookmark toolbar is hidden when Firefox goes into fullscreen mode. It’s quite annoying because I use the bookmark toolbar a lot. And since I use i3 window manager, I also use the fullscreen mode very often to avoid resizing the window. After some googling I found this quick solution on SUMO (Firefox commuity is awesome!).

The idea is that the Firefox chrome (not to be confused with the Google Chrome browser) is defined using XUL. You can adjust its styling using CSS. The user defined chrome CSS is located in your Firefox profile. Here is how you do it:

Open your Firefox profile folder, which is ~/.mozilla/firefox/<hash>.<profile_name> on Linux. If you can’t find it, you can open about:support in your Firefox. and click the “Open Directory” button in the “Profile Directory” field.

Create a folder named chrome if it doesn’t exist yet.
Create a file called userChrome.css in the chrome folder, copy the following content into it and save.


@namespace url("http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul"); /* only needed once */

/* full screen toolbars */
#navigator-toolbox[inFullscreen] toolbar:not([collapsed="true"]) {
   visibility:visible!important;
}

Restart your Firefox and Voila!

Rust Code Completion with YouCompleteMe

Fri, 13 May 2016 15:00:00 +0000

Every programmer knows that code completion can give you a productivity boost. Without code completion, you’ll have to read the documentation (or StackOverflow?) every time you forget what to type next. Since I’m a vim user and I worked with Rust very often, I’m glad to know YouCompleteMe (YCM), a powerful code completion plug-in for vim, officially supports Rust code completion (see this blog post). I’ll walk you through the process of installing YCM on Ubuntu/Linux Mint in this post.

Prerequisite

You’ll need:

vim (obviously!)
git (Vundle needs it)
latest stable rustc and cargo (YCM needs to compile the racerd completion server, follow this installation guide)
Python 2
Install the required packages for YCM: sudo apt-get install build-essential cmake python-dev python3-dev

If you use other OS, you can find the installation guide here.

Installation

First, we’ll install YouCompleteMe through Vundle. YCM is not a simple vimscript or python based vim plug-in, there are compiled components so you’ll have to compile it when you first install it, and re-compile it every time you update. Vundle is a package manager for vim; It allows you to specify the plug-ins you want in your .vimrc file and it will handle the install/uninstall for you. Vundle can be installed easily by following the official quick start guide.

After you installed Vundle, you can add the following line to your ~/.vimrc:

Plugin 'Valloric/YouCompleteMe'

Then run :PluginInstall in vim (in normal mode). Vundle should download and install YCM into the ~/.vim/bundle/YouCompleteMe folder. YCM is quite big in size, so if the :PluginInstall step may takes a long time based on your network connection, be patient and wait for it to finish. If you already have an old version of YCM installed, you can run :PluginUpdate instead.

Now, let’s compile YCM and include the Rust support:

cd ~/.vim/bundle/YouCompleteMe
./install.py --racer-completer

The install.py command It will take some time to compile. The --racer-completer argument tells the YCM installer to download racerd via Cargo and compile it for you.

If you ever wonder why it’s called --racer-completer, it’s because YCM relies on racerd for Rust semantic completion. racerd is a JSON/HTTP server powered by racer, a Rust code completion tool. (You can install racer directly and use it with vim plugins like vim-racer, but it’s not as convenient as YCM.) When you type your Rust code in vim, YCM will communicate with the racerd to get the code completion suggestions.

For the Rust completer to work, you’ll also need a local copy of the Rust source code. You can git clone the Rust source code from GitHub or download the tarball. Choose the version you’ll be writing Rust code in. Then you’ll need to point to the source code in your ~/.vimrc like so:

" In this example, the rust source code zip has been extracted to
" /usr/local/rust/rustc-1.8.0
let g:ycm_rust_src_path = '/usr/local/rust/rustc-1.8.0/src'

Now we are all set! Next time you opened a Rust file, YCM will start to work.

Usage

YCM automatically kicks in when you type :: or .. Here are some example:

Code completion for use:

Code completion for functions:

As you type more, the list of suggestions will be filtered automatically:

It can also help you “jump” to the definition of a variable, function, struct, etc. Simply locate your cursor on anything (in normal mode) and run :YcmCompleter GoTo to jump.

Let’s say we want to know the definition of inline_mode_assign_inline_sizes():

Boom! It jumps to the function definition:

You can map the verbose command to some hotkey, for example \] (backslash followed by right square bracket), add the following line to your ~/.vimrc:

nnoremap <Leader>] :YcmCompleter GoTo<CR>

Visualizing Performance Data on Perfherder

Sat, 07 May 2016 13:20:00 +0000

At Mozilla, we use Perfherder to visualize our performance testing data. Perfherder is a dashboard for monitoring and analyzing performance metrics, which can include page load time, JavaScript benchmark score, build time, installer size, etc. Perfherder allow you to visualize the data as a line chart, compare performance metrics between versions and automatically create alerts for sudden change in performance metrics.

Recently I was doing page load performance testing for Servo, our goal is to test how fast Servo can load real web pages, comparing to other browser engines. I build a test runner in Python from scratch. But for the data visualization and analysis, I want to levarage existing solutions. I tried Raptor, which is a dashboard we used for the B2G OS project. It was quite neat, using a influxDB backend and Grafana frontend. But since the B2G OS project is sunsetting, the Raptor tool is no longer under active development. So I decided to use Perfherder, which is used heavily by the Platform (Gecko) team.

Basic Concepts

Before we dive into the code, we should understand some basic concepts about Perfherder in general. Perfherder is part of the Treeherder project, which is a dashboard for test results. For the Gecko project, the test results are from build jobs and test jobs running on buildbot (deprecated) and TaskCluster.

You can see the architecture of Treeherder in this diagram. It’s OK if you can’t understand everything at one glance, we won’t need to understand them all to use Perfherder. For user who wants to submit test results, there is a web API server that accepts the test result data (as JSON), and Treeherder stores it in a MySQL database. When the user opens https://treeherder.mozilla.org, the web server renders the dashboard and the Perfherder graph from the data in the database.

Treeherder/Perfherder is for build results and test results, so it introduced the concept of a Resultset. A Resultset is a container for all the build results and test results triggered by a certain GitHub pull request or Mercurial push. A Resultset should contain a revision, which is the commit SHA for the top-most commit of the push. Since a push may contain multiple commits, the rest of the commits goes into the revisions list, which we shall see in the following code examples. And you can optionally add the commit’s author, timestamp and other metadata to the Resultset.

After the Resultset was created, we can add test results to it as a Job collection. The revision in the job collection should match the one in the Resultset, otherwise the result will not be visible on Treeherder. The job collection contains metadata associated with a test, like its name, the platform it runs on and the time it takes, etc. And most importantly, the job collection can contain artifacts, which can be test result data, logs, and any other artifact the test produced. For submitting data to Perfherder, we need to provide a performance artifact, which is an artifact with a specific format that Perherder can parse. We’ll show you the format in the following example.

Treeherder provides a REST API and a python client, so you can easily submit data to it programmatically.

So to sum up, a general workflow will be:

When someone opens a new push to the GitHub/HG repository, build and test it.
Collect the build metadata (commit hash) and test result, format them as Resultset and performance artifacts.
Submit the Resultset.
Submit a Job collection containing the performance artifact to the said Resultset.
Go to Treeherder/Perfherder to see the results.

Setting up a local Treeherder server

If you are submitting new kinds of data to Treeherder, you may want to host a Treeherder server on your own computer for testing. If you already have access to the stage or production server, you can skip this section.

Treeherder is open source (of course), and it provides a Vagrant configuration for you to install it effortlessly. There is an excellent documentation you can follow, so we won’t go into detail in this post.

After you installed the server as a Vagrant virtual machine (VM), you need to generate an API credential so you can push data to it through the Python client. After you are confident in your data submission process, you can request for access to the stage or production server. Notice that before you generate the credential, you should at least login to your Treeherder server once from the Web UI, otherwise the create credential script will complain that the user does not exist.

If you want to create a new repository on Treeherder, just like we do for Servo, you can follow these instructions.

Installing the Treeherder Client

Installing the Python client is easy. If you use virtualenv, you can simply run:

virtualenv venv
source venv/bin/activate
pip install treeherder-client

Then, you can import thclient in your python script to use it. It provides endpoints for Resultset and Job collections. You can read more in the official documentation, but be warned, the example code has errors. We’ll show you a modified example instead.

Submitting a Resultset

First, we need to submit our build metadata as a Resultset. The code we use for production can be found here.

The data should look like this:

dataset = [
            {
                # The top-most revision in the list of commits for a push.
                'revision': 'ab12d8098fcc0517b64643d25683e2e15e665410', # Commit hash

                'author': 'bos-servo <bors@allizom.com>', # Commit author

                'push_timestamp': 1462525798, # Unix epoch time

                'type': 'push',

                # a list of revisions associated with the resultset. There should be
                # at least one.
                'revisions': [
                    {
                        'comment': 'Auto merge of #11033 - mbrubeck:max-log, r=metajack'
                        'revision': 'ab12d8098fcc0517b64643d25683e2e15e665410', # Commit hash
                        'author': 'bos-servo <bors@allizom.com>', # Commit author
                        'repository': 'servo',
                    }
                ]
            }
        ]

Then, we fill in the data into a Resultset object for submission:

dataset = [
from thclient import (TreeherderClient, TreeherderResultSetCollection)

# Create a Resultset collection
trsc = TreeherderResultSetCollection()

# Fill-in the fields
for data in dataset:

    # Get a Resultset instance
    trs = trsc.get_resultset()

    trs.add_push_timestamp(data['push_timestamp'])
    trs.add_revision(data['revision'])
    trs.add_author(data['author'])
    # trs.add_type(data['type']) # Optional

    revisions = []
    for rev in data['revisions']:

        tr = trs.get_revision()

        tr.add_revision(rev['revision'])
        tr.add_author(rev['author'])
        tr.add_comment(rev['comment'])
        tr.add_repository(rev['repository'])

        revisions.append(tr)

    trs.add_revisions(revisions)

    trsc.add(trs)

Finally, we create a Treeherder client and submit the data (through an HTTP(S) POST request):

dataset = [
# Use the client id and secure you created before.
client = TreeherderClient(protocol='https', # Use 'http' if running locally
                          host='local.treeherder.mozilla.org',
                          client_id='servo',
                          secret='xxxxxxxxxxxxxxxx') 

project_name = 'servo'

# data structure validation is automatically performed here, if validation
# fails a TreeherderClientError is raised

client.post_collection(project_name, trsc)

This is how it looks like on Treeherder:

If you use a GitHub webhook, you should be able to get the commit metadata directly. But if you are running the build and test locally, you can use this script to dump the latest commit as JSON, so your Python script can load it with json.load().

Submitting a Performance Artifact

After the Resultset was created, we can submit a Job collection containing the performance artifact:

dataset = [
    {
        'project': 'servo',
        'revision': 'ab12d8098fcc0517b64643d25683e2e15e665410', # Commit hash
        'job': {
            'job_guid': job_guid, # This is a job ID from your test runner, use it to identify the test. 

            'product_name': 'servo',

            'reason': 'scheduler',
            'who': 'Servo',

            'desc': 'Servo Page Load Time Tests', # Test description
            'name': 'Servo Page Load Time',       # Test name

            # The symbol representing the job displayed in
            # treeherder.allizom.org
            'job_symbol': 'PL',

            # The symbol representing the job group 
            'group_symbol': 'SP',
            'group_name': 'Servo Perf',

            # If your test runner tracks the start and end time, you can submit it here
            'submit_timestamp': 1462525798,
            'start_timestamp': 1462525798, 
            'end_timestamp': 1462525798,
            
            # You can submit states like 'pending' so user can see the status of the test
            'state': 'completed',
            'result': 'success',

            'machine': 'local-machine',
            'build_platform': {
                'platform': 'linux64',
                'os_name': 'linux',
                'architecture': 'x86_64'
            },
            'machine_platform': {
                'platform': 'linux64',
                'os_name': 'linux',
                'architecture': 'x86_64'
            },

            'option_collection': {'opt': True},

            # jobs can belong to different tiers
            # setting the tier here will determine which tier the job
            # belongs to.  However, if a job is set as Tier of 1, but
            # belongs to the Tier 2 profile on the server, it will still
            # be saved as Tier 2.
            'tier': 1,

            # the ``name`` of the log can be the default of "buildbot_text"
            # however, you can use a custom name.  See below.
            # You can upload your test log to some public URL and point to it
            'log_references': [
                {
                    'url': 'TBD',
                    'name': 'test log'
                }
            ],
            # The artifact can contain any kind of structured data
            # associated with a test.
            'artifacts': [
                {
                    'type': 'json',
                    'name': 'performance_data',
                    # 'job_guid': job_guid,
                    'blob': perf_data  # Will be covered later
                },
                {
                    'type': 'json',
                    'name': 'Job Info',
                    #'job_guid': job_guid,
                    "blob": {
                        "job_details": [
                            {
                                "url": "https://www.github.com/servo/servo",
                                "value": "website",
                                "content_type": "link",
                                "title": "Source code"
                            }
                        ]
                    }
                }
            ],
            # List of job guids that were coalesced to this job
            'coalesced': []
        }
    }
]

After you submit this data (see the submission code below), you’ll see a test result marked as PL in the SP group:

In the data, notice that we didn’t define the job_guid, this it whatever unique ID you used in your test runner to identify a test. We didn’t have this in our simple test runner, so we just randomly generate a string for it.

In the artifacts section, we submitted two types of artifacts, first is the performance artifact, which we’ll explain right away. The second is a job_details artifact, which will show up in the Treeherder UI like so:

The performance artifact perf_data should look like this:

dataset = [
"performance_data": {
    # You need to mark the framework as "talos" so 
    # Treeherder will generate a "Performance" tab
    "framework": {"name": "talos"}, 
        "suites": [
        {
            "name": "domComplete",
            "value": 3741.657386773941,
            "subtests": [
                {"name": "about:blank", "value": 1000},
                {"name": "163.com", "value": 14000},
            ]
        }
    ]
}

Here we submit the domComplete time for loading the pages. The value is a geometric mean of all the subtests’ domComplete time. You can use other ways to summarize the results from all the subtests. Be careful not to submit data that is over 80 characters long, Threeherder has an undocumented limit for that.

~~By assigning the framework as `talos` (a performance suite used in Gecko)~~

, the Treeherder UI will generate a “Performance” tab for us, from which you can find the link to the Perfherder view.

Edit: If you have a new perofrmance test framework, and want Treeherder to generate the “Performance” tab for you, you should open a bug for that. Here is an example. After you add your framework to treehderder/model/fixtures/performance_framework.json, you can use "framework": {"name": "your_framwork_name"} in your performance artifact.

By clicking on the test result number 9607, you’ll be directed to the graph:

If you click on the “+ Add more test data” button, you can choose to show all subtest data:

And this is the code for actually submitting the data to Treeherder; we leave it to the end because it’s similar to the previous one. This is of course a simplified version, to see the real code I run in production, check here.

dataset = [
tjc = TreeherderJobCollection()

for data in dataset:

    tj = tjc.get_job()

    tj.add_revision(data['revision'])
    tj.add_project(data['project'])
    tj.add_coalesced_guid(data['job']['coalesced'])
    tj.add_job_guid(data['job']['job_guid'])
    tj.add_job_name(data['job']['name'])
    tj.add_job_symbol(data['job']['job_symbol'])
    tj.add_group_name(data['job']['group_name'])
    tj.add_group_symbol(data['job']['group_symbol'])
    tj.add_description(data['job']['desc'])
    tj.add_product_name(data['job']['product_name'])
    tj.add_state(data['job']['state'])
    tj.add_result(data['job']['result'])
    tj.add_reason(data['job']['reason'])
    tj.add_who(data['job']['who'])
    tj.add_tier(data['job']['tier'])
    tj.add_submit_timestamp(data['job']['submit_timestamp'])
    tj.add_start_timestamp(data['job']['start_timestamp'])
    tj.add_end_timestamp(data['job']['end_timestamp'])
    tj.add_machine(data['job']['machine'])

    tj.add_build_info(
        data['job']['build_platform']['os_name'],
        data['job']['build_platform']['platform'],
        data['job']['build_platform']['architecture']
    )

    tj.add_machine_info(
        data['job']['machine_platform']['os_name'],
        data['job']['machine_platform']['platform'],
        data['job']['machine_platform']['architecture']
    )

    tj.add_option_collection(data['job']['option_collection'])

    # for log_reference in data['job']['log_references']:
    #    tj.add_log_reference( 'buildbot_text', log_reference['url'])

    # data['artifact'] is a list of artifacts
    for artifact_data in data['job']['artifacts']:
        tj.add_artifact(
            artifact_data['name'],
            artifact_data['type'],
            artifact_data['blob']
        )
    tjc.add(tj)

client.post_collection(project_name, tjc)

It’s Your Turn

This is how easy it is to submit data to Perfherder. If you have any project relevant to Mozilla, and want to show your data on Perfherder, you can request for access to the stage or production server. Since everything is open source, you can also consider hosting your own Treeherder/Perfherder instance. If you are interested in Treeherder and Perfherder, you can read their official documentation, report issues, or contribute code to their GitHub repository.

If you are interested in Servo’s performance testing, you can follow this GitHub Issue. Submit issues and contribute pull requests in the servo-perf repository. (Although we might soon merge the code back to the Servo main repository.)

Special thanks for William Lachance for teaching me how to use Perfherder and opening accounts for me.

Shing Lyu - Articles

Lessons learned in writing my first book

The Pomodoro Technique works!

An Outline is important

Version control and LaTex

Use a grammar checker, multiple times

Get the book now

Download JavaScript Data as Files on the Client Side

Counting your contribution to a git repository

Number of commits

Count the line of insertion and deletions by a user

Other alternatives

New Rust Course - Building Reuseable Code with Rust

What’s this course about?

What’s so special about it?

A bottom up approach

A guided tour though the std

Conclusion

Chatting with your website visitors through Chatra

How it works

What I love about Chatra

Easy setup and clean UI

Considerations for bloggers who can’t be online all day

Customizable and programmable

Out-of-the-box Google Analytics integration

Pricing and features

Conclusion

How to Unit Test WebExtensions

Mocha and expect.js

Testing asynchronous code

Conclusion

如何貢獻開源專案？

心理準備

基礎知識

找尋可以貢獻的 bug

如何下載、修改並測試 Servo 的原始碼

如何提交 Pull Request

哪裡查資料與問問題

其他人的貢獻經驗

Merge Pull Requests without Merge Commits

Minimal React.js Without A Build Step (Updated)

1. Load React.js from CDN instead of npm

2. Get rid of JSX

Using 3rd-party NPM components

Modules with browser support

Modules without browser support

The benefits

Taking notes with MkDocs

Using MkDocs

Migrating from TiddlyWiki

Conclusion

Make LastPass Work Across App and Website

Porting Chrome Extension to Firefox

What needs to be changed

Publishing the extension on AMO

Install Ubuntu 16.04 on ThinkPad 13 (2nd Gen)

Some assumptions

Before Installing Ubuntu

Installing Ubuntu

What works?

What doesn’t work?

More on setting up media keys for i3 window manager

Conclusion

An Overview of Asia Tech Conferences in 2017

How to Contribute

Vim QuickFix for Rust and Servo

rust.vim

Servo

Beginner's guide to git rebasing and squashing

Fixing review comments

Rebasing

Fixing conflicts

Squashing

Ask for help

Mutation Testing in JavaScript Using Stryker

Introduction to Mutation Testing

Mutation Testing from JavaScript

Setting up the environment

A simple test suite

Running the mutation testing

A guided tour though the `std`