I was setting up the XAMPP package on a Windows 7 box and wanted to configure some virtual hosts. This is fairly easy – there is a dedicated configuration file for virtual hosts and in my case they needed to be virtual virtual hosts – so I added them to the local Windows hosts file and to the Apache configuration file and saved.

Restarting the server gave the error code “-1 server not started”. One really frustrating fact about XAMPP on Windows (7) is that the errors in the configuration which prevent the server from starting is not written to the error log. Instead (and this might be the preferred behavior of others) it is logged using the Windows logging mechanism which is stored/found gods-knows-where?!! What is up with that?!

After finding the (right) log which actually showed the error nicely – I found that it was my “Order” setting that caused the trouble. It turns out that it has three valid values “allow,deny”, “deny,allow” and… some other setting I don’t care about… I had happily typed “allow, deny” and this simple mistake prevented the whole server from running! A tiny whitespace. One tiny whitespace!!

When going over the configuration to spot errors I never even thought of this as being an option – I am now much wiser.

Please pardon my ignorance – but I stumbled upon an article in Dr.Dobb’s – about “Dataflow programming” – which is actually what this is all about! If you would like to learn more on the matter or refresh your knowledge please go read the article – it describes the problem and solution very well. You might also want to read the Wikipedia article on Dataflow programming and the one on Flow-based programming. If you have more time and thirst for this subject please read this article which clarifies why this type of problem solving is suitable for parallel hardware.

I wrote a blog post containing the october figures at the Southbound Software blog – and before that a post containing the september figures. I believe that the october figures are more representative than the september figures – although september rocked hard!

Recently I’ve read some blog posts on the general Android hardware/software design. Especially this post and the mention on DF triggered some emotion!

My point is that I really don’t think that Android missed the target with the hardware back button. Not at all. I actually think it is the most useful button.

The thing is – I use both Android and iOS on a daily basis – and I’m not sure that is the case for many people. When I am using my Android devices I really use the back button a lot and when I use my iOS devices I keep complaining that it is not there.

The example given in the article mentions the Twitter app. Let’s all agree to leave the Twitter app out of this discussion – it is clearly broken. When I get a notification about a new mention on Android and press it the app fails to load the tweet. The fact that I can’t get to the Twitter app main screen is not really a problem with my hardware – it is the result of bad app design!

When I use my Android phone I usually read news (actually I just process news feeds and send relevant articles to Instapaper using the Read Later for Instapaper app). When I get an e-mail, text message or someone mention/writes me on Facebook I get a notification. Pressing the notification takes me to the app handling the notification and when I am done – I press the back button – and by magic I am back in my news processing cycle – amazing!

When I actually read news – which I usually do on my iPad – I use the Instapaper app. It is very nice – but when someone e-mails me e.g. I get a notification (luckily I’m describing the iOS 5.0+ flow!!!) and press it – which takes me to the app (so far not very different from the Android story). The trouble starts when I want to get back to the article I was reading. If I press the software back button in Twitter/Facebook/Mail/etc. I go to someplace inside the app! If I want to get back to news reading (and trust me – I really need to get back there!) I have to press the button (no need to call it names – there is only one) which takes me to the home screen where all my apps are (wtf!!). Here I need to locate the Instapaper app – it might be on another page (it is where I left it though – but WTF?!) – and press it to re-open the program. Alternately I can choose to double-press the button (see this flowchart for more information – seriously WTF!!!?!!) and get the list of recent apps and choose Instapaper.

As you can see – I clearly need to adjust my use. Trying to apply “the Android way” on iOS gets me frustrated. Using “the iOS way” on Android is just a silly waste of time.

To me this isn’t really about what mobile OS is right or wrong! iOS delivers a very strong single-running-application frame that can be used to very tightly control the user experience without thinking about any other app than your own. And iOS does this very effectively and very good. Android on the other hand is made to display activities. An activity is a single screen in an application. Activities stack and the hardware back button takes the user backwards through the stack. This is very powerful – especially considering the ability to mix activities from different apps on the activity stack!

Android and iOS handle stuff very differently. This is because they are two very different operating systems – made for not-so-different devices. They each handle their specific task very nicely – in my humble opinion. The real problem is lazy conversions – the Twitter team should clearly have done a better job understanding the two platforms instead of trying to make a one-to-one conversion.

As I said in the beginning – I like Android and I like iOS – and I love developing apps for both!

The rating system only has one mayor flaw!

Most users don’t acknowledge that the apps are moving targets. Most apps are updated frequently in response to user comments and suggestions. Therefore user comments and ratings must change to reflect this.

As an example: I wrote the Wake on LAN Client and released it to the Android Market. It quickly got a few ratings and some comments. In the description of the program I specifically stated that comments/suggestions should be e-mailed to me directly as there is no way to comment on the market. On july 26th the user Geronimo gave a one star rating and posted the comment “Keeps force closing when WiFi is disabled to save power…”. A few days later when I got the time to narrow down the problem I fixed it and released an updated version of the app. The problem is that the user comment + rating is still there – anyone checking out the program could think that this is still an issue and therefore not download the app.

The Wake on LAN Client is free and I made it to meet some personal needs. I would guess that the problem is even larger for paid apps!

My advice to people who comment and rate is: Please update your comments and ratings to match the current state of the app. Please don’t make comments or ratings based on crashed and/or flaws in the app before contacting the developer by e-mail – I am sure they will help you and possibly update their app to fix the issue.

Let’s say you have a spreadsheet. It consists of N rows and M columns. The columns 0…M – X are raw data – they most likely come from a different source but thats really not the point. The rest is the interesting part. They are calculated using formulas – most people have done this a million times in spreadsheets – it’s really not that big a deal.

Lets say I have a column C1, it is calculated using a formula. So C1 = C2 + C3. It’s a not-so-complicated formula – but that is not the point. I have many calculated columns. And they all use other columns in their formulas – all these formulas need to be calculated to show me the full dataset of N rows and M columns. This means that the program making the calculations are making (a minimum of) N * X calculated cells.

See – humans are very organized when they work with spreadsheets. Usually we don’t clutter things up all that much. Which means that column CN usually aren’t calculated by a formula using any column CM where M > N. This comes very natural – we mentally execute from left to right (western-style) and don’t really like forcing our minds to do otherwise.

This leads to the normal execution style. We execute the calculation of new columns from left to right doing on at a time. Which is very time consuming. So now we have a new approach – which isn’t exactly rocket science – but it’s worth considering when you develop applications like ours.

Given a set of columns that needs to be calculated using formulas that contain reference statical columns we build a dependency graph. We apply a neat trick at the beginning. The static data is assembled in one node. If a formula reference any column on the static data we make a dependency to that node. There are columns that A: Don’t reference any column (these are rare – at least in the software that we develop), or B: Don’t reference any static column – which is very common for calculated columns referencing only other calculated columns. Consider the following dependency graph:

Here node #1 is our base data – we can’t and won’t do anything about that one! Node #2 is a calculated column using a simple formula utilizing the data provided by node #1. Node #3 and node #4 are based solely on the data provided in the column created by node #2. Node #5 represents a column created with the input of column derived from node #3 and #4.

To deliver a full dataset to the user every node must be processed. This is the time consuming part and therefore we are looking for ways to speed up this process. In this case we have nodes #2-5 that needs processing (remember the base data in node #1 is static). The simple way to deliver a better-than-sequential result is to move processing of individually independent nodes to parallel execution units. In fact – my solution to this problem is to actually start a calculation thread per node. Each of these threads will be waiting on the threads calculating the dependencies of the node. By doing this the execution will run in parallel – and hopefully this will result in a performance gain considering the overall execution speed of the complete graph calculation. On the graph presented node #2 will be the first to get processed. It’s only dependency is the root – which needs no processing. Right when the calculation of node #2 is done the threads controlling processing of node #3 and #4 will start in parallel – these two only have one dependency which has already been processed. Finally – when both node #3 and #4 are done – the calculation of node #5 will begin, as both #3 and #4 are dependencies of #5.

Keeping a graph structure is a also a great benefit in other ways. An example is how it allows one node to be changed – forcing a re-processing of all dependent nodes by having a recursive “dirty” marking scheme.

The process is driven by a long list of new features – this is how it always starts – but the overall design goal is “better and more simple”. This goal is of course extremely difficult to achieve. Combining a lot of new features with the extensive list of ones already in the software – while making the overall experience better and easier takes a lot of hard work.

One place where we have done an effort is on concept list. We have come a very long way with our demo of the new version – but the new features introduced some new concepts – like “formulas”, “policies” and “calculated columns”. These concepts – or what they represent – is needed in the software – but having a complete list of words and lingo to learn when starting to use our software made for a very steep learning curve. Especially considering the fact that two of the three new concepts was totally hidden in like 80 % of the program – which is bound to make users forget they even exist!

So – we’ve done the only thing we found really worked – we reduced the number of concepts the user has to juggle! From three to – one! “new column” – it is very easy to understand – the user can do exactly the same thing as before – and the code only needed a little tweaking to make everything with one concept. Not to mention the UI – which simply needed to be reduced to work the new way!

Hopefully our users will feel the simplicity when the program hits the streets!

Our main product resemble a spreadsheet application with many custom data views and functions. On previous versions of the software we have executed the calculations sequentially with no real performance issues rising – but as we try to push the boundaries of our software we are aware of the limits that we may encounter.

Specifically large datasets and complex dependent calculations would make performance suffer! A side note to this is that previous versions of our software has handled the data storage and manipulation 100 % in memory – which naturally sets a limit in size e.g. – the program is running as a 32-bit application (due to COM-dependencies) and crashes occur at around 1.4 GB memory usage!

The new version has this stuff all sorted out. We simply use an on disk database for storage. This makes all data creation and manipulation very expensive! Especially compared to the previous version – duplicating a .NET DataTable object don’t cost nearly as much time as duplicating a SQLite table on disk. This naturally calls for some performance tuning.

Speaking in general terms – the desired performance is reached by limiting database operations to a minimum and executing them in transactions. Furthermore the performance is tuned by building a dependency graph and executing operations in parallel.

What is really interesting is the way that different people choose to focus in the process of – well – anything related to “software”.

First thing I did with this particular site was to copy the database from one host to another. After that I moved the files (FTP download, FTP upload) and finally I changed the code (PHP) to reflect the new host – some database settings and some mail settings. At first load the page looked great – everything actually looked right. Navigation revealed no mistakes or configuration errors! Great!

Then I/we moved along to test some signup sequence (involving mail sending) and then problems started appearing! When listing some of the database content we saw that headlines suddenly contained question marks! Ouch! And the mails that the system should be generating never reached their target! Ouch again!

My natural response was to start debugging the mail send sequence – this part relied heavily on web server and SMTP server configuration – something had to be wrong somewhere. But the reaction from my peers was to focus mainly on the question marks in the text!

This situation is very common – I see it at least one every week at work – developers focus on the real problems, the real errors – and designers and managers focus mainly on the visual issues. I mean – who really cares if a line of text is placed two pixels higher or lower! The actual trouble is that these cases usually takes a very short time to fix and this leads designers and managers to think that it is better to fix right a-freaking-way! It is not! Cases like these are best solved by grouping them together and solving all of them at once – you know – that way you’ll have a whole hour worth of work.

When a dialog (yes/no, ok/cancel etc.) is shown in Windows (at least in Windows 7) you can actually copy-paste the dialog text and get some meaningful output. The following output comes from the SQLite Database Browser “do-you-want-to-save” dialog, check it out:

—————————
SQLite Database Browser
—————————
Do you want to save the changes made to the database file C:/Users/Michael Banzon/Documents/Stage5.db?
—————————
&Yes &No
—————————

It is very nice! It actually produce som understandable output even for non-techies – I can imagine that I’ll be using this feature frequently now that I know that it is there. Pasting text into (Gmail) e-mails is so much easier than pasting images!