SQLite DB with Orange Data Mining

The following post has no agenda or moral. It's just a story about stuff I did at work and some code I published.

Not a long time ago I was requested to bring my data-mining knowledge back from the dead. The project is actually quite fascinating, something in the software security field, and it's a shame I cannot provide much more details in this blog.
Anyway, since budget is low, and I tend to prefer open-source solutions, I installed Weka and Orange on my machine. Since the raw data to be processed is stored in a SQLite DB and since I recall I had better experience with Weka - I went for the first.
After a few hours of breaking it and trying to get it to produce some decent classification rules, I gave up on it. I guess that the fact I now love Python more than Java (it wasn't always like that) had quite a serious weight on my decision to stop trying to get Weka to work, so Orange it is.

Unlike my previous experience with Orange, back when I was a student, I figured I should go with the core framework - no UI, only Python commandline and scripting using Orange's extensive data-mining libraries.
After refreshing my memory with the tutorials, I felt comfortable with it, yet I realized Orange isn't capable for handling SQLite DBs. Instead, it uses some of the industry's common file formats for data-mining, and TSV. Yes, TSV. But this shouldn't stop me, right?
Few minutes later, I had my first SQLite-to-TSV converter up and running. You can find the sources on github.

From here, everything was a breeze: some data fiddling, some algorithms threshold settings, and results started to appear. Cool.

Boring Android UI

I am an Android user for a year and two months now. The only Android device I have ever owned is the Samsung Galaxy SII. Before that, I had installed nitdroid on my N900, but it was mainly for fun and the ability to show off how great the N900 was. My only previous experience with smartphones UI was with devices made by Nokia: E65, N95 and as mentioned, the N900. The first two had:

• A home screen with a single row of the most useful apps, similar to the bottom row usually found in modern smartphones.
• A menu of all of the other apps, which is basically a grid of icons and folders containing icons. Exactly like what can be found in modern smartphones.

The N900 was more refreshing, as it supported widgets, kind of like what you can find on Android devices (hi Apple, people love having widgets, seriously).

So when I got my Android device, I felt at home right away with the new desktop environment, which originally was the TouchWiz launcher. Actually, I can't understand Apple's claim of Samsung copying its UI from them, since it reminded me much more of the Nokia UI, but never mind...

After playing around with the device for a while, I figured I should be switching to other launchers, to get a richer experience of what can be done with the Android desktop. But seriously, other than effects and perhaps smarter widgets, there was nothing more to it.

Recently I tried Launcher 7, which is supposed to mimic the Windows Phone experience on Android devices. Since I have only one friend who owns a Windows Phone device (and he is a MS employee, so you cannot credit him for actually buying the device), I wasn't very familiar with the Windows Phone UI. After a using it for a while I was really impressed. This is a really fresh smartphone UI, which doesn't resemble anything else out there, and most importantly - doesn't resemble the UI I had back in 2007.

Does it mean I suggest you all to start using Windows Phone? no. It does mean that there is place for innovation which I hope will happen as soon as the new Windows Phone devices will start gaining market share.

Update: in the week that past since I wrote the first draft of this post, I've demonstrated Launcher 7 to several other Android users and all of them were really impressed as well. I guess there's something to it...

Blogging for the .NET framework

A system I've been working on is based on the .NET framework. It seems that Mono and Apache on a web server is a pretty good combo, and I recommend everyone with a spare day or two to give it a try.

Anyway, I needed to run a blogging engine on that same server and have that blog support Hebrew and right-to-left languages. While there are several good blogging engines for .NET, not all would run on Mono and most doesn't do the RTL part good enough.
My first attempt was to use SubText, but getting rid of the SQL Server dependency and adding the RTL support became too much of a hassle. I picked SubText for it being notorious for its simplicity and elegance, but while I was ready to invest some customization time, it demanded too much of it.
My second attempt was BlogEngine.NET. What can I say - I fell for it almost immediately. The platform is versatile enough to support not only many RDBMS's, but it also allows storing the posts in XML files. Nonetheless, the translation to Hebrew wasn't complete and there was no RTL native theme for it. So I made one.

BlogEngine.NET is hosted on CodePlex and uses Mercurial for its source control. I had a fork of the project in no time, so I started making the required changes and created a new theme based on the default one. Since I don't want to maintain a fork of the project for eternity and since I want others to enjoy this new feature, I've sent a pull-request which was accepted yesterday. So starting from version 2.5.0.6, you can enjoy BlogEngine.NET for your Hebrew blog.

As always, it feels good to contribute back and to know that my code can be further enhanced by other developers.

Happy blogging.

Starting Mono Development In No Time

For anyone coming from the Linux world, getting up and running with a development machine is a matter of minutes. All one has to do is use his favorite package manager in a manner similar to this:
apt-get install mono monodevelop mongodb python...
And just sit back and wait.

But what do you do in those days when you have to use Windows for development? What if using a Linux VM isn't a viable option?
Enter Chocolatey.

Chocolatey is a superb open-source package manager based on NuGet which brings the benefits of package management, including keeping your software up-to-date, to Windows. Due to its core, I believe that it was initially targeting .NET development, but this is not the case anymore. Among the packages I install are GnuWin (because you can't live without the GNU command line tools), node.js, mongodb and python. Obviously, one can use it to install Mono and MonoDevelop.

I believe that with the great community that has started to evolve around Chocolatey, the repositories will grow richer and into more areas, and Windows development would become as fun as Linux development.

Enjoy.

Creating Wireframes with Pencil

When working on a project which requires a user interface, it is usually best not to jump ahead and start programming the UI part before making some sketches first. It makes sense, since UI development shouldn't be different than library/API development. The client is different, but the principals remain.

For a long time I've been a Balsamiq Mockups person. I know, it's not open source, but they'd used to give licenses to those who blogged about it. Also, few years ago, Balsamiq was far superior to other solutions and it contained much needed features.
This is not true anymore. For the past few months I've been using Evolus Pencil and I can tell you it's great. First - it's open-source. Second - it has ports to all platforms and can also run as a plugin for Firefox. Moreover, it is customize-able, extensible and very flexible.

So if you've decided to give Pencil a try, here's a tip that would make your wireframes look a bit more serious and professional: right click the "Sketchy GUI" collection in the collections tab, and select "Collection Settings...". Navigate to the "Text" tab in the resulting dialog, and change font family from "Comic Sans" to a font that you might actually use for the final product.
Want another tip for general wireframes design? Don't ever use "loren ipsum" as a text filler. Wireframes should be later translated into real UI and graphics design, so you don't want to find out that that text area you've been designing doesn't fit the needs of your application. Always wireframe with real data.

Those were my two cents for wireframes using open-source software. Hope it helped.

Using App.Config in IronPython

I love python very much, and as you probably already noticed, I love C# as well. I already have many libraries and utilities developed in those languages, and for me, the ability to reuse code between those languages, is a must.
So I came to know IronPython, which is a complete Python implementation for the .NET platform, using the DLR. For most of the time, the integration between the two is a breeze. All you have to do is "clr.AddReference" your .NET assemblies, and you're good to go.

Unfortunately, with more complex projects, there are some caveats. Most notably is the missing ability to load app.config (or web.config, or any other .config) files from IronPython. Without diving too much into details, the reason is the way AppDomain works in .NET. It is looking for [assembly name].config file when one is requested using the ConfigurationManager. Since IronPython code is invoked with ipy.exe, the configuration file must be called ipy.exe.config. I suppose the same goes with IronRuby as well. This is a serious limitation.

Thanks to Tom E. Stephans for providing this awesome proxy for the ConfigurationManager. Basically, his code injects some logic to the inner workings of the ConfigurationManager, that allows us to specify which config file to be loaded. This config file will be seamlessly loaded by the ConfigurationManager when needed.
Unfortunately, Tom's code doesn't exactly compile. Probably due to some copy-paste issues he had when publishing it. So I did some minor modifications, and removed the part of the code responsible for loading custom configuration sections and uploaded it to Bitbucket, so everyone can use it. The code is available here, along with a sample usage python script.

Few things to keep in mind regarding my code:

• It was compiled using Mono 2.10 like this: mcs ConfigurationProxy.cs -t:library -r:System.Configuration
• It was tested both under Mono and under MS .NET framework 4.0.
• There's a sample config file called blogsample.config which contains a demo connection string. This configuration file is loaded in test.py and the connection string is printed.
• Just like the ".ConnectionStrings" collection is accessed, one can access the ".AppSettings" collection.
• Since the injection is done to the ConfigurationManager, implicit usage attempts will also work. For instance, the SubSonic ORM which requires a connection string, will work seamlessly.

Once again, kudos to Tom for his great work. I hope you will find my corrections and sample useful.