maanantai 7. toukokuuta 2012

There is something wrong with OSX Java 1.7 2D performance

While doing performance optimizations to my small Thrust-style game project grambers I noticed something really weird: when I upgraded the JavaVM from 1.6.0_31 to 1.7.0_04-b21 the frames-per-second halved.

As grambers is scala based project I first suspected that the problem is related to java-scala version incompatibility. I pretty quickly ruled it out by measuring where the extra time was spent: in Java's graphics2D.drawImage. I then tested the Java2D official example DukeAnim by just adding a measurement around the paint.drawImage call and got the following results on my MacBook Pro:

reynders$ java -version 
java version "1.6.0_31" Java(TM) SE Runtime Environment (build 1.6.0_31-b04-415-11M3635) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01-415, mixed mode) reynders$ java DukeAnim 
50 draws took 19ms
50 draws took 23ms
50 draws took 24ms
50 draws took 29ms


reynders$ java -version 
java version "1.7.0_04" Java(TM) SE Runtime Environment (build 1.7.0_04-b21) Java HotSpot(TM) 64-Bit Server VM (build 23.0-b21, mixed mode) 
reynders$ java DukeAnim 
50 draws took 226ms
50 draws took 219ms 
50 draws took 222ms 
50 draws took 217ms 

10x reduction in speed! I ran the same tests on a ages old Lenovo T60p using Java 1.7.0_04:

java -version 
java version "1.7.0_04" Java(TM) SE Runtime Environment (build 1.7.0_04-b20) Java HotSpot(TM) Server VM (build 23.0-b21, mixed mode) 
java DukeAnim 
50 draws took 23ms 
50 draws took 33ms 
50 draws took 25ms 
50 draws took 27ms 

So the problem seems to be in the Java 1.7 OSX implementation. As of writing oracle does not accept OSX bug reports in their java bugzilla and the macports site is down I decided to create this blog post as a way to help people with the same problem. I will report a bug to Oracle when possible - if the problem does not go a way before that :)

A dragonfly on a rock in Khao Sok Rainforest, Thailand

maanantai 30. tammikuuta 2012

Why Open Source Development style requires Open Source Software projects

No matter what your preferred software development ideology is, a project with no management, no process and no co-location should sound like a recipe for disaster.

Still Linux Kernel developers have succeeded in developing a very large and complex software with minimal overhead. Most of the development is done by mixed set of individuals who rarely physically meet each other. Their work is pulled together by similarly distributed individuals called maintainers and eventually by Linus himself.

Very successful open source software (OSS) development has been done with an even simpler setup: someone just sets up a (D)VCS repository (GitHub etc) and starts publicly developing software. Random people get interested in the project and start contributing by sending the original author patches or even pushing changes directly to the repository. In many cases all that is needed to coordinate the work is a common (D)VCS, few emails and/or chat sessions - and of course the source code itself.

If distributed development works this well, could and should it then be used to do non-OSS software? Here the term "non-OSS" refers to the organization and the nature of the project, not to the license of the software. So hire the best professionals from around the globe and offer them good compensation. Create common repositories for code. Have product owners, architects and managers feed the developers with specifications and requirements so that they deliver something that has business value. Set up bug tracking systems and continuous integration machinery to solve the quality problems. Create custom development and testing tools so that developers can maximize their output. Open Source Development on steroids, right?

Unfortunately, no.

The motivation to do Open Source Software development is very different from the non-OSS development. The magic that makes OSS development successful can only happen in a true open source project. Open Source Software development works without management, process and co-location because OSS developers are interested in the project they work for. As Daniel Pink pointed out in (http://www.ted.com/talks/dan_pink_on_motivation.html) no amount of money can beat this.

Pretty much everything else that is the OSS magic comes from this single fact:

OSS developers work on items which they themselves feel are the most valuable and interesting things to do right then. If somebody disagrees about the priorities then either they get into a consensus or the task is just dropped and a better one picked. No one is working on a task they do not believe in.

OSS developers know that anyone can and potentially could review their code or design. Ever noticed the manic urge to clean up your house when visitors are expected? Public review gets the better half of you going.

OSS developers know that unless they themselves communicate and synchronize between the other involved people they will be screwed and most likely people you value will know about it.

OSS developers pick development tools that work best for them. And if they are not happy with the best available tools they will make their own.

The same applies for other aspects of open source software development: have bunch of motivated people with same goals in life work together and the magic will start to happen. In a project where somebody else decides what should be done or who should do what you most likely will not see similar benefits. You will see unmotivated people creating stuff they do not believe in, people not communicating with each other and all kinds of quality and schedule problems.

My recommendation is obvious: if you want to do software in an OSS style then let the people decide what to do and how to do it. If you want to control the content or output of the project then apply other methods. I would start with having all the people co-located but that is another story, then ;)

Orange Lantern, Espoo, Finland

keskiviikko 20. huhtikuuta 2011

"Build borken!!!" - Adding voice to your build radiator

As in any good ContinuousIntegration obedient team we have a web page showing our current build status. Anyone lurking in our team space sees the build state all the time from a monitor. Additionally if there is a failing build an email is sent to everyone.

Unfortunately the build takes a long time to finish so the state changes are rare and people receive lots of emails so a broken build is often not noticed for a long time (my long time means more than a few minutes).

To cure this we added voice to our radiator: if the state of the build changes it tells it verbally. As we were not able to change the build system or the radiator we did it by an external ruby script.

Main ingredients of the script:
  • The ruby http-uri open("url").read returns the radiator web page as html string
  • The ruby string.scan(/regex/) can be used to look for the build status from the page
  • http://www.rubular.com/ helps to figure out the needed regexp<(li>
  • "echo something | espeak" in Ubuntu Linux and "say something" in mac os x says something :)


With these it took around one hour to brew in the needed ruby script. Now whenever our build state changes we really notice it! Here is our script as an inspiration to anyone interested (if the regexp looks bad no worries as it was easy to come up with by trial and error using Rubular).
A window shutter in Garmisch-Partenkirchen, Germany

keskiviikko 16. maaliskuuta 2011

How I use git in an open source project

I neither digged nor "got" git for a very long time.

I used to use subversion both professionally and in my own projects and had really hard time figuring out what could be the benefit of git's apparent added complexity. This complexity reminded me of Clearcase which had done nothing but made my life miserable whenever I was forced to use it. Thus when I started contributing to both Malva and later Jato, which are both github based projects, one of the first challenges was to start using and, yes, even understanding Git.

Although the internet is full of more or less thorough git tutorials the truth is that because of its flexibility it can be a totally different beast from project to project and from developer to developer. Please keep this in mind when reading any git material, including this blog!

You should probably have a glance at some git tutorial first, but if you are like me, pass the dull reading and just start using it. Siberia teaches you!

So here is how I use git to develop Jato.

Setting up git to work with Jato


Install git.

Tell git your name, email and signature acknowledgement:


git config --global user.name "My Name"
git config --global user.email "my@email.address"
git config format.signoff true


You need to do this only once.

Then you need a copy of the Jato repository on your computer.


git clone https://github.com/penberg/jato.git


(you'll get the github project clone url from the project's front page).

By default you are now working in the master branch where you should never ever work in! Thus lets create a work branch and switch there with:


git checkout -b my_work_branch


You can check which branch you are in with:


git branch


and switch between branches with


git checkout branch_to_switch_to


Anyway lets stay in my_work_branch. Here you will do your changes, hopefully not that many. Why? Jato people prefers to develop in small clean increments. It makes the review process and synchronization between people a lot easier. For example when I refactored a 250+ function interface I did it in about ten independent separate batches of work over many weeks (Git calls these increments as patches but well get back to that in few moments). Just keep in mind that it is better to do the work in too small than in too large increments.

Once your have made the increment just throw the branch away, those are dirt cheap in git!

Doing the work


Where were we? Oh yes, so we are about to start work in my_work_branch. First make sure Jato build is green with


make check


If all tests pass you are green to go doing changes and adding new files. If you add a new file and want git to know about it just


git add *my_file.abc


When making changes you should periodically try to get the build to pass. Usually it is also a good point to make a commit to your local clone:


git commit -a -m "A brief but descriptive commit comment"


Repeat and rinse until you have achieved a small increment. Maybe you have added a single unit test and the piece of functionality that it tests, great! Time to throw it to the wolwes, i.e. to Jato maintainer.

Patch


As mentioned earlier git has a concept called patch. Forget any binary patches you might have encountered when updating applications or operating systems. In git world a patch is simply a textual output of diff your_version some_other_version and a few extra lines on top of that. Luckily git does most of the work for you.

To do a patch from the latest new commit in your branch first make sure that you have the latest changes from the github master:


git checkout master
git pull
git checkout my_work_branch

(and if there were any changes:)

git merge master


You must resolve and commit any merge conflicts. Once done and the build is green you are ready to create the patch:


git format-patch -s HEAD^


This creates a text file called something like your-commit-comment.patch. If you only did a single commit on top of master you are done. If there were more than one commit you must squash them into a single commit. There are many ways of doing this. I prefer this way:


git checkout master
git checkout -b my_merge_branch
git merge my_work_branch --squash
git commit -a -m "My only commits commit message"


And now you do the patch exactly the same way as previously shown:


git format-patch -s HEAD^


Now just email the patch to a Jato maintainer!

Looking back at this blog entry reminds me why I thought git was complex :). Fortunately and unfortunately the only way to learn git is to use it! In hindsight it didn't take that long to learn the essentials, although maybe the fact that I had a linux kernel maintainer helping me made it a little bit easier. Luckily all Jato developers have the same benefit!




Nuuksio National Park, Espoo, Finland, Dec 2009. The real Narnia

maanantai 14. maaliskuuta 2011

Why I want to do Open Source development?

Few words about my background and why I decided to do open source development. No big revelations about making software here, but maybe a familiar story to some of you ;)

I've been doing software for nearly two decades now. It started as a hobby as me and my friends needed helper utilities to play a complex role playing game called Rolemaster (truly a great game by the way! It has had a markable effect on my views of team work).

Anyway I started with Pascal, then got excited about "real" game development and learned assembly and, as the usual gate theory goes, ended up coding in Ansi-C. At the same time I started to work in the IT industry and study CS.

After a couple of years coding C both professionally and as a hobby Java came to an existence and I moved on to that. Strangely, although I coded and did "architecture" work (I'm little bit allergic to that word nowadays) for many years with great enthusiasm I somehow ended up being Agile Software Development Coach. If this came as a some sort of a surprise to my dear reader you are not the only one ;). The best explanation I can come up with has been that I could not and cannot code if I see that it is the utter waste of my time and my client's money.

Due to this coaching work I have had less opportunity to code at work. That's why I've lately spent time learning new languages and frameworks like (ruby, RoR and scala mainly) and trying to find open source projects to participate in.

Currently my main effort is contributing to Jato and Malva.

The reason why I "chose" these is that the creator of this hotspot virtual machine and a hotspot testing harness happens to be my colleague and a friend. They also happen to require both Java and ansi-C knowledge and in the future even more exiting stuff so they turned out to be a really good playground for a generalist like me.

Malva, being basically a core java API compliancy test harness was a really good excuse to get to do tons of unit tests and to have to dwell into the less known parts of the java API. It was also a nice sandbox to learn the open source development process without having too much other pressure.

Jato on the other hand is less than trivial project. Although I did my master's thesis on Java Virtual Machine internals I would still say that I do not know anything about how Jato works and what its architecture is (ask Jato creator Pekka Enberg for that ;). Thus to document my learnings and to communicate with other people interested in OSS or Jato development I plan to blog about my travel into this domain. I expect to learn something about modern Open Source development and maybe even succeed in contributing something useful to this promising project...maybe ;)


Mangrove trees, Daintree rainforest, Australia. Really good in containing tsunami waves, btw

lauantai 12. maaliskuuta 2011

Mounting a NAS with user write permissions

I recently bought a NAS, Lacie Network Space. Although being slower than for example any USB mounted hard drive it works quite well as the common storage between my ten or so different networked devices I have at home.

The only big problem I had was that the drive was mounted as read only on my linux system. I could sudo to get write permissions but it only helped on command line operations. From GUI it was read only. Darn.

I know nothing about mount so I had to bang my head against a wall for quite some time. Googling for help made it even more confusing. I finally gave up and read the manual for mount and figured out how to do it.

Automatic mounting is based on configuration on /etc/fstab. The original entry for my NAS was


//netshare1.local/OpenShare /media/netshare1_openshare smbfs guest 0 0


To get it to mount as writable by my normal user id reynders I had to change it to this:


//netshare1.local/OpenShare /media/netshare1_openshare smbfs guest,uid=reynders,gid=users 0 0


Reboot and voilá!

Koh Phangan - Pretty much at the exact same time as the 2004 earthquake and tsunami hit the islands on the other side of the Thailand peninsula few hundred kilometers away. Notice all the crap that the full moon party people have left on the beach

Me and the blog

I've done software development for couple of decades. I aim for good quality code and sensible working methods.

Lately I've done less coding and focused more on coaching software development teams and organizations. To keep my skills in relevant shape I've been trying out new programming languages and also participated in various open source projects.

As I'm also an enthusiast environmentalist and a traveler.

This blog is about both my hobby and work experiences. I'll also throw in photos and videos from my travels.

I'll start with a green sea turtle I spotted while snorkeling in Akumal Bay, Mexico, February 2011. This was just one of the many we saw so it was truly a magnificent place!