2014年6月8日星期日

A developer's App directory on OS X

As a linux guy, it's a mixed feeling to use OS X in daily work:
  • bare the low disk performance when enjoying the slickness of beautiful UI
  • executing bash script inside VM when listening music by Spotify
  • hardly speak in OS X shell utitlies like launchctl, ipfw after getting used to upstart and/or systemd
  • you do want tweak darwin kernel which is a different world from sysctl.
  • keep confusing your thumb between alt and command key.
In a nutshell, I DO love linux, but OS X is so beautiful and slick to interact with her. This blog post is about those apps keeps me stay in OS X. If you're not a developer, I would suggest reading this post from lifehacker to find more for other apps: The Lifehacker App Directory: Mac OS X.

Communication

iMessage Let me know if you can find a better one with simplicity.

Mail client: Sparrow Lite supports gmail feature very well like labels, importance, etc.


IRC client: Lingo is the best qualified but free IRC client I can find.


PhoneBox Lite lets you make, receive, record phone calls on OS X. you just need to connect your mobile phone over bluetooth. handsfree. You may not need this app once you own iphone plus OS X Yosemite in this fall.

Productivity

Feed Reader: Feedly is the desktop version of famous online feed reader.


Pages, Numbers, Keynote : free office suite. you can google it on how to get it for free on OS X 10.9.

Text editing: Sublime Text 3: This is the best text editor after using textmate, gedit, ultraedit, texttable, vim (but not Emacs). Atom is a good alternative but it needs time to catch up with sublime in all aspects. I almost forget there is such thing like IDE in developer's life.
Update: I'm using Atom since 0.135.0 which is decent enough. Although it's not as smooth as that from Sublime, those git plugins are awesome. BTW: atom is free while sublime needs $69


Alfred 2: You will need this to enter keyboard-driven era. You may not need this on OS X Yosemite, but it's super extensible and I will keep using it.


Dash is my favorite app to checking manuals of all different kinds of tools, programing languages and utilities when offline.


Markdown editing/viewing: Mou is one of the best free markdown editor.

Utility

Homebrew is "the missing package management tool for OS X". better than macports, you will fall in love with it if you like apt-get or yum.

Dropbox allows you share files across all different platforms from android, Ubuntu, iOS to OS X.

VNC viewer: Screen sharing is a hidden app within OS X. run command ln -s /System/Library/CoreServices/Screen Sharing.app within your /Applications directory to get it in launchpad.

Text to speech: say is an awesome command line utility to convert text into audio. try $ say hello world in your terminal :)

Virtualbox: Install via command brew cask install virtualbox if you need Linux, Windows, Android VM on top of OS X.

Git tool: SourceTree: I use gitx as well but prefer this one for multiple project and view all kinds of history of git commit.


Followup

I suggest reading the book Mac OS X for Unix Geeks to learn more tips on the difference between OS X and Linux.

As you may know after reading my previous two blog posts Dual Boot Max OS X and Fedora19 on MacBookPro and Power saving on MacBook Pro runs Fedora 19, I run Linux on mac hardware. there are enough reasons to do this and I will list those apps keep me in Linux on mac later.

2014年5月7日星期三

Undocumented features of grunt-mocha-webdriver

There are always neat packages in developer communities to help you as long as you can find them. Today my example is grunt-mocha-webdriver. Yes! you get it (from the name): it can help if you use gruntjs, mocha and selenium webdriver in your web application project or testing project.

Getting started

Make sure you want to keep reading if you:

  • want to test your web applications
  • manage child processes for testing in Gruntfile.js and it's a boring job
  • maintain custom test driver, like injecting environmental options into test suites
  • want to have options on selenium service providers
Or just read the ****ing source code directly if you're rock star developer and jump into the code section directly. 

Note: I'm not providing free test minutes backdoor to saucelabs.com nor browserstack.com...If you ever think about it, please go to their popular free trial page.

Read doc

Follow the readme file of grunt-mocha-webdriver to pick it up in minutes. Then keep reading.

Check out additional features

#1: I like saucelabs.com...but I dislike creating tunnel (it doesn't work all the time)
You can use the following options to treat saucelabs.com as a normal selenium service provider plus online videos of test session:


grunt.initConfig({
    mochaWebdriver: {
        saucelabs: {
           options: {
               hostname: "ondemand.saucelabs.com",
               port: 80,
               identifier: null,
               browsers: [...],
               ...
           }
        }
    }
});


When means it will connect to saucelabs.com but no tunnel created. Don't forget exporting your username and access key to environment.

Yes, you can, I'm not kidding and it works very well. in the same way as #1, you need to provide the configuration information (no tunnel created):


grunt.initConfig({
    mochaWebdriver: {
        browserstack: {
           options: {
               hostname: "hub.browserstack.com",
               port: 80,
               identifier: null,
               browsers: [{

                   browserName: "internet explorer",
                   platform: "WINDOWS",
                   version: "10.0",
                   "browserstack.user": process.env.BROWSERSTACK_USER,
                   "browserstack.key":  process.env.BROWSERSTACK_KEY
               },
               ...
               ]
           }
        }
    }
});

Then you can enjoy the full list of browser/platform combinations from browserstack.com.

So far, you still need tunnel to test application started at local machine, I didn't have good experience on testing over tunnel yet.

Finally, don't blame me if your Gruntfile.js becomes too long :)

2014年4月14日星期一

Feedback information from WebDriver/Selenium

WebDriver is one of the best tools to test web application automatically with real web browsers. it works great until:
  • test session timeout
  • uncaughtException happens
  • explicit wait doesn’t work

Usually people don’t know the next action because it’s hard to get useful feedback information especially it’s a selenium server in a linux box without x server environment. I mean, you can’t see them at all.

Which kind of information is useful

As a web programmer, typically the following information would be helpful no matter you use WebDriver or not:
  1. capture a screenshot - usually it tells what is on the page you tested. that’s it. Maybe you can find where the unexpected result if you’re lucky.
  2. logs of test session - it tells you what the webdriver does to browser, like “webdriver is moving cursor to x,y”, or “locate element zzz” etc. not fun for non-webdriver fans.
  3. browser console - shows javascript errors. this is useful.
  4. network activities - you just need to see the network panel of developer tools, right?
  5. screencast of your test - I like movies! it does tells how it loads the test scenario on browser. Usually people like me who get the screencast will forget #1 at all.


Keep reading if you want these information or you want to see how to do them in Node.js.

Assumption

It runs a selenium standalone server with xvfb in a linux box.
Write test in Javascript with WebDriverJS, run in Node.js. Don’t tell me you are learning yet another language to test a language. that’s just weird.
Firefox. I’m not a fan, but it is easier to work with, seriously.

Screenshot!

This is a out of box capability of webdriver. so here I just show how to work with screenshot (like save to local filesystem):

    var webdriver = require(“selenium-webdriver"),
          fs = require(“fs”);

    webdriver.promise.controlFlow().on("uncaughtException", function() {
        webdriver.takeScreenshot().then(saveScreenshotToFile);
    });

    var saveScreenshotToFile = function(data) {
        var fileName = Math.random().toString().substring(2);
        var screenshotDir = getEnvValue("SCREENSHOT_DIR") || process.cwd();
        var filePath = path.join(screenshotDir, fileName + ".png");
        fs.writeFileSync(filePath, data, "base64", function (err) {
            if (err) {
                console.error(err);
            } else {
                console.log("saved a screenshot to %s”, filePath);
            }
        });
        return filePath;
    };

when you want a screenshot during your test, just use:

    webdriver.takeScreenshot().then(saveScreenshotToFile);

Developer tool panel (console log, network panel, etc)

It provides you the information in developer tools of Firefox just like when you developing web app.
The trick is Firefox profile. and you need to customize it. Fortunately, there’s a ready npm package named “firefox-profile” to help (Thanks to the author @saadtazi). So you just need to use the following code when you start the selenium server:

    var FirefoxProfile = require(“firefox-profile”),
        webdriver = require("selenium-webdriver"),
        SeleniumServer = require(“selenium-webdriver/remote”).SeleniumServer,
        serverOptions = {args: [“-Dwebdriver.log.file=/tmp/firefox-console.log”]}; // dump javascript console log to a file

function buildWebDriverWithFirefox (serverURL) {
    var profile = new FirefoxProfile(),
        defer = webdriver.promise.defer();
    // set network monitor as selected tool in developer tool
    profile.setPreference("devtools.toolbox.selectedTool", "netmonitor");
    profile.setPreference("devtools.netmonitor.enabled", true);

    // webdrvier test session log
    profile.setPreference(“webdriver.log.file”,”/path-to/test-session.log”);
    
    profile.encoded(function (encodedProfile) {
        var driver,
            capabilities = webdriver.Capabilities.firefox();

        capabilities.set("firefox_profile", encodedProfile);

        driver = new webdriver.Builder()
            .usingServer(serverURL)
            .withCapabilities(capabilities).build();

        defer.fulfill(driver);
    });

    return defer.promise;
}

new SeleniumServer(“/path-to-selenium/selenium-server-standalone-2.38.0.jar”, serverOptions))
        .start()
        .then(buildWebDriverWithFirefox)
        .then(function(driver) {
             // post-action to driver object if you need
             // for example, make sure the developer tool is open
             var openDevTools = webdriver.Key.chord(
            webdriver.Key.CONTROL,
            webdriver.Key.SHIFT,
            "Q"
          );
          return driver.actions().sendKeys(openDevTools).perform();
        });

Screencast!

Yeah, this is the most exciting one: you can watch what browser is doing, in a video. Actually there’re many solutions if you google it, and those solutions do work, and almost all of them depends on ffmpeg, which is an awesome tool for dealing with video/audio.
The design is pretty simple:
  1. start xvfb to simulate a graphic environment
  2. run browser process in it
  3. capture x11 display as input for ffmpeg
  4. output them to somewhere.


That’s all. Usually it fails to record screen because:
the screen size mis-matches. so you need to make sure specifying screen size and color depth for xvfb-run process like:

$ xfvb-run -s -screen 0 1024x768x16 -a mocha …

Then start ffmpeg process like:

$ ffmpeg -s 1024x768 ..

Here’s a working recorder which capture screencast in .mov file which uses mocha test name as file name:

// screen-recorder.js
var fs = require("fs"),
    path = require("path"),
    spawn = require("child_process").spawn;

/**
 * the screen recorder behavior is controlled by 2 environment variables:
 *
 *  - SCREENCAST_DIR - determine where the screencast files are saved.
 *  - SCREENCAST_SAVE_PASSED - weather saving screencast for passed tests
 */

module.exports = function (test) {
    var api = Object.create(null),
        movieFile,
        recordingDir = process.env.SCREENCAST_DIR,
        recorder;

    api.start = function (done) {
        if (recordingDir) {
            movieFile = path.join(recordingDir, test.fullTitle() + ".mov");
            recorder = spawn("ffmpeg", ["-y", "-r", "30", "-g", "300",
                "-f", "x11grab",
                "-s", "1024x768",
                "-i", process.env.DISPLAY,
                "-vcodec", "qtrle", movieFile]);
        }
        done();
    };

    api.stop = function (done) {
        if (recordingDir) {
            if (recorder) {
                recorder.kill();
            }

            if (test.state === "passed" && !process.env.SCREENCAST_SAVE_PASSED){
                fs.unlink(movieFile, done);
            } else {
                done();
            }
        } else {
            done();
        }
    };

    return api;
};

Then use it in mocha test:

    var screenRecorder = require(“./screen-recorder”), recorder;

    beforeEach(function (done) {
        recorder = screenRecorder(this.currentTest);
        recorder.start(done);
    });

    afterEach(function (done) {
        recorder.stop(done);
    });

However, it’s boring to transfer .mov files across build server, S3/FTP and laptop. actually I just want to see the video, don’t want to download it at all… So let’s use streaming instead of dumping videos on file system to watch live test!

Thanks to ffmpeg which means streaming video extremely simple, Here’s what I did before running any mocha test:

before(function() {
    spawn("ffmpeg", ["-y", "-r", "30", "-g", "300",
                "-f", "x11grab",
                "-s", "1024x768",
                "-i", process.env.DISPLAY,
                "-f", "mpegts", "udp://192.168.33.1:1234"]
    );
});

Which means it sends video to address 192.168.33.1:1234 over UDP streaming. you can use any video player supports UDP streaming to watch it. I use VLC and it works great. just open File -> Open Network … then fill in URL `udp://192.168.33.1:1234`.

Tips: open UDP port 1234 on your laptop:

$ sudo ipfw add 7000 allow udp from any to any dst-port 1234

Next steps

It’s still not that easy. so I may spend some time on:
deliver all those capabilities into an example project on github.com
deliver the configuration into a docker image so you can get a running selenium server with all those configuration.

send screencast to a streaming service so that you can watch it afterwards.

2014年3月10日星期一

Use cucumber.js in browser without Node

This is just a detailed guide from question Running Cucumber.js in Browser where people asked followup on my answer.

Usage

Here's the steps on how to run the example with node runtime:

    $ git clone https://github.com/cucumber/cucumber-js.git
    $ cd cucumber-js/example
    $ npm install
    $ node server.js
 
Then open your browser and navigate to http://localhost:9797/, click the button Run feature. you should see the test result in green background.

Now you would think you must depends on node to use cucubmer.js, but you will find nodejs just make you get the live result easier, NOT a mandatory runtime dependency.

Resources

Start with server.js, it's a connect application to host a web server so that you can access the example over http. all resources you get over http can be saved as static content and you can host them with any web server:

  • /index.html - static file
  • /example.css - static file
  • /vendor/* - static files
  • /example.js - static file
  • /cucumber.js - this is the key resource and yes, it can be saved as static file.

/cucumber.js

In a nutshell, it's compiled javascript. cucumber.js is designed for nodejs runtime where the dependencies are managed by npm, the example code is introduced to use browsers in this commit. no surprise, it uses browserify.

You can get compiled cucumber.js for browsers by the following command:

    $ cd cucumber.js
    $ node -e "var str = require('./bundler')().bundle(); require('fs').writeFileSync('./cucumber.js', str);"