Valgrind is a memory checker and dynamic analysis program, named after a Norse myth. So the obvious question is how to pronounce “Valgrind”?  The “grind” is pronounced with a short ‘i’ like “grinned” (rhymes with “tinned”) rather than “grined” (rhymes with “find”). One of the big bonuses is Helgrind, a tool for checking race conditions in multithreaded code. It is still ‘experimental, but you now how ‘experimental’ is when it comes to open source codes.

    I am trying to integrate Valgrind with my unit testing framework, but I haven’t had the time. For Raydo, I develop on OSX, and run the Valgrind & unit tests on Linux, since Valgrind only works on Linux. This will catch Linux memory leaks and memory leaks from my code, but not any memory leaks that are specific to the Window platform or the GUI kit that you are using. Those leaks are the platform’s problem anyway. You will see a lot of memory leaks from other parts of you software , including wxWidgets, when you run Valgrind on Raydo. Such is life.

    Valgrind supports Linux on x86, x86-64 and PowerPC. There are ports to other the *BSD Unixes. There is no port for Microsoft Windows nor are there any plans for one. Valgrind works much better with C++/STL than with straight C. Valgrind does not check ranges in arrays, such as strings in C, which can be a problem.

So if you code C++/C on Linux, I highly recommend Valgrind.

WikiOverlords
Valgrind home page

    Well, the build process is going well. On the mac, I have made a shell script that creates the applications for you. For those of who who have not developed on a mac yet, mac applications are actually disguised folders that hold the application along with other information on running the app. Sure, it looks like just an app if you are looking at it in the finder, but it is really folder. Before, we had to build with X-Code ( the free gcc development kit from Apple Computer ) using project files and such. I really did not like going from the command line and make files to xcode JUST to make the application I could click and run. So now there is a new command in the makefile to build all the apps, or you can build the apps by doing a make in the application name and adding ‘.app.

Now onto windows … again.

    Compiling with pthreads-win32 has tons of gotcha’s. Really read the faq on the website. Some of the things I really need to stress is that if you are using an old version of MingW to develop on windows, please get the latest version. Older versions have ‘weirdness’ when it comes to pthreads. And by weirdness, I mean crashes.

    I have to hand it to the developers for putting pthreads ( POSIX 1003.1-2001 ) on Windows. They had to build it on top of the Win32 event model which is not the best. Windows NT, 2000, and XP models are different from Windows Me and the Windows 9X series. Do I think Microsoft will ever put a stake into this things and release a nice built in POSIX pthreads library that runs on all the above platforms? Not likely. Windows Me and the Windows 9X series are pretty close to dead anyway, so just do the NT kernel.

    Now Microsoft does have a pthreads implementation in Microsoft Windows Services for UNIX Version 3.5 from Interix. That runs on NT based systems, and is basically UNIX running inside of a Windows NT based box. Supposedly they have support in SUA 5.2 that I could use to build against. But I won’t. Feel free to try it yourself, but I am going to stay away from the kool-aid. If you are going to install Microsoft Windows Services for UNIX , which is basically a UNIX box, why not just install Fedora or Ubuntu and call it a day?

I know ‘Databased’ is not a word. But it was the best word I could come up with. What I am doing now is connecting my unit test output to a MySql database.

By putting the unit tests, I can look back and see when things failed, which help when I pull old files out of svn. So I can sort by pass/fail test ( which is the most common method ) , the test performed and the class.

This image above is RUnit.php . I will include this file in the php/ subdirectory. It only has three parameters to modify at the top of the file and that is about it.

I use a simple homegrown unit test framework. Yet another C++ unit test framework, like we need another one. I needed a much less specific framework. But a very simple framework.

I use cron to do automated tests. I pull the latest copy out of svn, compile run the unit tests, and store the results in the database. It is pretty sweet, but I do not have the failed compiles feeding into the database yet.

About three years ago , the guys from the OCC got togethter and decided to make changes to the way options are quoted by the market. For an example, The Mar 08 120.00 Puts have the symbol QAAOD while the Mar 08 125.00 puts have the symbol APVOE. So after a few years of work, they came up with this :


Mar 08 120.00 Puts
QAAOD -> AAPL080322P00120000
( AAPL 08 03 22 P 00120 000 )


Not really too difficult to figure out the coding on this system. There are a lot of details concerning things like FLEX options and stocks splits/mergers etc. The big change is due for July 2009, so a about a year and a half we will have to move over. The old symbols will be suppressed after that date.

What does this have to do with the price of tea in China? Not much ( but you can check the tea futures contract ) and not much to do with Raydo.

Internally, we already use the new structure. Just an input filter to change the old symbols into the new symbols. After the big change, we will just drop the filter.

I am testing out changing the build system to CMake. I want to make building Raydo as simple as possible. Right now I use make, which is not the friendliest way to build things. By using CMake, I can supply , for example, Visual C project file.

So to summarize:
Open Source
  Supports basically every UNIX, MS Windows (MSVC, Borland, cygwin, mingw) and Mac OS X
  Can generate Makefiles and *projects* for KDevelop3, MSVC 6,7, XCode
  Has no other dependencies except for a C++ compiler

So I will be able to supply Windows guys with project files.

    So I was dorking around this weekend with the threading model on Raydo. Mainly benchmarking the granularity of the threads. For what I normally do , everything seems fine, equities, futures, blah, blah,blah. Until I started simulating a market maker carrying tons of options quotes on his/her book. The old model started to have a ‘lag’ that I was not happy about. So I started benchmarking a finer/coarser grained threading models.

    I am using pthreads which run on real ( POSIX ) systems, and Pthreads-win32 for that other operating system. So I am looking through the pthreads documentation for pthreads read / write locks , in the Linux/Mac/rest of the world documentation, I see this.

To prevent writer starvation, writers are favored over readers.

Which is nice, and IMHO, the way reader/writer locks should be. However for Windows, I see this lovely pearl of wisdom,

Pthreads-win32 does not prefer either writers or readers in acquiring the lock – all threads enter a single prioritised FIFO queue. While this may not be optimally efficient for some applications, it does ensure that one type does not starve the other.

    Argh ! “optimally efficient”? How about sucks? I do not want a fresh tick quote to come in, and have 20 readers in the queue before the writer reading a stale tick quote. I want the writer bumped to the front of the queue, dump the new tick in, and have the readers read the new data. Another reason I don’t like windows.

    Pthreads is just a wrapper around Win32 threads, and from what I am reading, Win32 does not really support multi-read, single-write (MRSW) lock that does not have problems with writer starvation. ( In NT they have an undocumented lock, which I guess is only for some nefarious MS usage.) So don’t quote me on the Win32 API. I am not a Win32 API guy, but it seems the Win32 API has a problem with writer starvation in reader/writer locks. Please correct me if I am wrong.

    Whatever model I use, when running a lot of reader / writer threads, I think I will have not have the warm fuzzies using Pthreads.

[Update] Just changed the object model a little bit, and everything is back to goodness. I did learn something about the Win32 API along the way that I did not want to know.

The Problem with Threads is a paper written by the Edward A. Lee, the Chair of EE @ University of California at Berkeley

I especially liked one line from his conclusion,

Concurrent programming models can be constructed that are much more predictable and understandable than threads. They are based on a very simple principle: deterministic ends should be accomplished with deterministic means.

Until that day comes, we will still muddle along with pthreads. Once again, threads in Windows is a pain!

( Well not really a pain, just why do they have to be different ? )

     I cleaned up the C++ interfaces a little bit. This image below is only for the C++ interfaces.

    I am concentrating on Postgres, sqlite, Oracle and MySQL. Postgres and SQlite have the most project friendly licenses, and MySQL / Oracle have the least friendly licenses.

    However, the cluster aspects of MySQL and Oracle are awesome. Something the other two do not have.

( I know Postgres has clustering, but it is not the same )

   So it only took….not much time to get a Raydo building under windows.

   It was very easy, much easier than I expected. Usually you expect some complication to crop up. Fresh installs are so nice, and worked perfectly this time.

    One caveat, read the instructions!! After installing VC++ 8, you have to make some configuration changes when installing the platform SDK. It boggles the mind that they (they as in M$) could not make those changes transparent when installing. That would make it much easier.

    So I checked Raydo at of the svn repository. and hit build. It compiled straight out of the box. Always nice when that happens.

    I do not know if Windows will be as supported as much as Linux or OSX. Actually I do know. It will not be supported as much. The difference is now, if there is a question or problem on windows, I can at least debug.

Blah. I still hate windows.

    Ok, to answer a question, the interface is written C++. NOT wxWidgets.NET. Chills run down my spine just thinking about it. And to answer the second question, the Alpha GUI will be ready when it is ready. Sorry, but I want to get the interface right the first go around ( or at least build in enough flexibility ) that I don’t ‘code myself into a box’ that would require me to break a lot of stuff to fix the UI.

    So here are some of the changes I have made. ( With screencaps ! )

Click to enlarge

[1] The rows are thinner so more information can be shown to the user.

[2] First cut at user defined views ( Notice the View 1/2 tabs )

[3] The Books ( or desks , or whatever you want to call them ) can be user defined.

[4] I took out the application menu for now. I want to roll part of the functionality into the main screen, and the rest into the status menu.


Design is not how something looks, but how it is used. -Unknown