I also got yesterday a Telit GM862-GPS module, breakout board (and antennas) I had gotten earlier. This morning I had little time to look at it and managed to read GPS position and time data, and both send and receive SMS using the serial interface. Next I'll try to write a small proof-of-concept program to reply to incoming SMS with the GPS time to test the built-in Python interpreter.

Also on the electronics front I've been trying to get gcc-arm working on my OSX to make some improvements to the firmware for this datalogger, the problem is that the firmware is built with Rowley crossworks (while thankfully not Windows only, "tested on RedHat 9" does not exactly sound like the bleeding edge of Linux support) which doesn't use Makefiles. So I dug around a bit for an example of a Makefile and the standard header files for LPC21xx uCs and got the original firmware source to compile, however when I program the uC with that hex file it won't boot.

Now this may simply be because I made some stupid mistake while tweaking the Makefile I found, if you have experience in writing Makefiles for LPC21xx take take a look (extract that to the directory where you already have extracted the firmware found behind the datalogger link) and let me know what is my mistake.

GM862-GPS update: the command reference manual is very thorough, without thinking I let it print on my big printer (LJ 8000DN) and some 600 pages later I noticed that all the command tables (syntax, description, output and example for each) are listed in full for each flavour or version of the interface (older ones are selectable for backwards compatibly on newer HW revisions), even when there is absolutely no difference. To add insult to injuru I ran out of toner so the last 100 or so pages are hard to read, and of course they're the ones that contain the important info for the latest and greatest version of the interface...

and then to the issue: if with Exorcist you run into

net.sf.exorcist.api.ContentException: java.sql.SQLException: No suitable driver
        at net.sf.exorcist.midgard.MidgardImporter.importContent(MidgardImporter.java:445)
        at net.sf.exorcist.core.Exorcist.main(Exorcist.java:127)
Caused by: java.sql.SQLException: No suitable driver
        at java.sql.DriverManager.getConnection(DriverManager.java:545)
        at java.sql.DriverManager.getConnection(DriverManager.java:171)
        at net.sf.exorcist.midgard.MidgardImporter.importContent(MidgardImporter.java:405)
        ... 1 more

That's all for now, I just needed to write this down so I can find it the next time I run into it.

I've also been working on the tag library bergie mentioned, if someone has spare time and good hand with JS a del.icio.us workalike widget for tag input would be cool, the search backend is already in exec.

The org.routamc.photosgtream component, (to replace net.siriux.photos in 2.6 MidCOM), is still very much a work in progress due to lack of time, hopefully this weekend sees some progress in that front.

• PuTTY
• Opera, either the mini (java based, free to use) or the Mobile (better, costs money)
• KeyPadLock, since the device itself has no provision for automatic keypad locking

Beverages revisited

I redesigned the thermostat circuit, reduced some complexity and made the sensor to be connected via a 3.5mm jack (signal on hot [the tip] , supply voltage on cold [middle] and ground on sleeve). The new circuit is attached as termostaatti.tar.gz.

And then some pictures from making a version of the circuit (it got then revised a few times with "green wire" [red in this case, but anyways]):

But still it was about as stable as Windows PC in a microwave, I decided to try and switch SIM card to a newer one in case it would help, in theory it should not have any effect as long as the card is not ancient, but I was running out of easy options and after making the minor monetary sacrifice things were in fact running much smoother.

This is not to say that it would be exceptionally solid, but when you start at the bottom all improvements seem major...

I have noticed that the camera optics and sensor are definitely optimized for video, and in fact I dropped the still image size to 1600x1200 since the largest size is just a waste of storage, the image stabilizer in the video mode OTOH works very well.

Fixing the car

Last saturday when going to lure coursing Taru noticed that the headlights, parking lights and meter lights were all out, and all fuses and realys were ok... So I got in contact with an aquitance who told me that the problem must be the actual light switch then (and that those were very "english" in their construction), he also told me that the official spare part is hideously expensive.

He came over to demonstrate the correct way to dismantle the instrument board, a quick test with a multimeter proved that the switch is indeed the problem. So I got one with a more sound design and a large diode.

Here's some photos (taken with the N93)

The original switch (not dismantled, it indeed is open like that):

The new switch, without and with collar and in place:

This land rover belongs to the guy in question:

Rebuilding the lab

The computer (10yr old Compaq Armada) finally gave up, so I took one of the old "spare" pc-laptops from the office, it already had Xubuntu installed and Eagle is available on Linux as well so I was happy, untill I found out that the Canon _LBP-800 I had for printing circuit boards is a true "WinPrinter" and it was not possible to get it working under linux. I took the HD from the old laptop but it wouldn't boot properly. Luckily we also had just laying around and old LaserJet 2100M which works just beatifully under Linux.

While getting the switch mentioned above I noticed that the electronics supplier had a soldering station on sale, I quick phone call to consult with my brother later it was decided that the family business definitely needs one of them...

"Design workstation", "The lab" (exposure box, development solution and acid) and the "Soldering workstation"

Update 2006-09-20: I've put some of this and some more to Midgard documentation, another page there also covers the Squid setup I promised.

Quick and Dirty things

in my.cnf make sure query cache is enabled:

query_cache_type = 1
query_cache_size = 10M
query_cache_limit = 1M

There are other caches to consider as well, see the example my-xxx.cnf -files, usually included in the install. Also tuning the caches to your specific situation may require some work.

Especially if your filesystem is slow for some reason, make sure the atime doesn't get stored for MySQL database files, if you have MySQL datadir as it's own partition (you should), then in /etc/mtab add noatime to the mount options and remount. If not you can use chattr to set the flags on the files via the following command chatttr +A -R /path/to/mysql.

More about it

Here's a few articles on optimizing mysql, O'Reilly has a good book about it as well.

• Optimizing MySQL: Hardware and the Mysqld Variables (esp the variables part)
• MySQL's Query Cache
• MySQL Presentations: Optimizing MySQL

With 1.8 a lot of things have changed so we should rethink the indexes Midgard uses, I and Piotras have this under work but suggestions on the dev list are welcome.

As a side note, if it's a public site you can also run Squid as reverse proxy in front of it making things a lot faster, I'll try to find time to write about it while I still remember the issues I ran into when doing it on two occasions in the last few weeks.

Other things

My N93 has been in repair for slightly over a week now, I guess it really had faulty memory (though the repair people have not yet confirmed anything else than that they had to send it to central repair, which suggests a difficult-to-fix HW problem), I cursed this on monday when going to training I saw a true classic parked next to the F-Secure office:

I had to get a third party plugin to get iSync to recognize it and after that I ran into the main annoyance so far:

I'm not sure if it's hardware or software(/firmware) issue but "heavy" memory access to the phones internal memory freezes the device...

It took a ton of reboots (by removing battery) and retries to get my contacts list synced over bluetooth but finally I managed, first I thought the bluetooth stack had issues but later when I tried to transfer a 6MB file to the internal memory via the USB cable it froze as well but transferring to the MiniSD card worked fine.

I got the nokia portable bluetooth keyboard paired with the phone but so far I haven't figured out how to get it actually working, the downloadable "drivers" (it's a software you need to leave running in the background) do not yet support N93.

Update

While playing around with the "Catalog" software (which is a software catalog system) I found out that it did have N93 compatible version of the wireless keyboard software, it's has some latency but is very usable, especially since I havent used "morse code" in years (my previous phone, P900 had touchscreen and on-screen keyboard, plus handwriting recognition...)

In other news

I finally got rid of the thing that has been choking my bike, the catalysator is gone and the injection system is loaded with more feisty software, the difference is certainly noticeable, especially at lower revs.

I also have been finalizing the so called universal chooser widget for datamanager2. this chooser is a generalized version of the contactchooser widget I made for datamanager1 and it also supports creating new objects if the target component provides handler for it, tomorrow I'll try to make a sample implementation of the creation handler to org.openpsa.contacts (for persons and groups).

The nokia bluetooth keyboard I had ordered ages ago had finally arrived so I picked that up and after installing the plugin it worked like a charm, since it folds just in two parts it's kind of largish compared to mechanically more complex solutions (like the old Palm keyboard), but not too large and the feeling of it is very nice for such a keyboard.

Also arrived just before me is the N93 which I haven't yet picked up but will as soon as I can, I'll post my experiences here.

Dragonbane

As you can see I didn't have any spare time for diaries at all, in fact due to various problems the SFX team was very short-handed and I didn't get to play that much either (two nights and a day in between, though the day was a very nice "enjoy the atmosphere of cinderhill" one).

But it was quite fun doing the FX as well (though you might consider hauling a ton of gear and pyrotechnics/explosives around in dark, rocky forest with hills less than that), and of course there's always something special in pressing that red button on the detonator.

Most importantly I didn't think of work at all, and thought of IT things only very little (compared to my normal time usage)

I met a lot of new interesting people, and can't wait to meet them again someday

For photos check out http://www.kuvateko.com/peek/ (some selected photos) and www.kuvateko.com/db/ (more photos organized by subject), all photos taken by Janne Björklunds team of professional photographers.

I only took photos from the return trip, which I still need to download from camera, sort out and then upload, I'll let you know when it's done.

The thing I missed most during all this was training (though in Timos wedding in the early hours me and Towo discussed many things related to it), but today is training again...

I'm now on a ship headed to stockholm, from there I ride to Älvdalen to prepare for the actual event.

True to traditions I finished my gear only this morning and good chunk of last night was spent making some of� it.

In case I have time to write down my musings about the pre-game hustle-bustle you can read about them in D Diaries.

This is also the start of my summer vacation, Taru will join me after the game and we'll go to a friends wedding nearby, then we'll probably start heading back to Finland via Ahvenanmaa and the archipelago (which I must to my shame I have never done before this, such beatiful nature and so near [it must be the "too near" syndrome, you just don't appreciate it properly unless you have to travel far to see it]).

The board, the speaker is attached directly to it (marked in red).

Backside of the board, the connectors of the speaker located.

Switch in place and speaker hot-glued to where it was (the contacts were insulated once the jumper wires were soldered in place), a few mm higher of the board (though there is plenty of space inside the case).

The thing back together.

Last week I got the new modem (Diamond SupraExpress 56e PRO) and started experimenting, more than slightly troubling was that under no settings of pvftormd could I get the modem to play voice files trough its speaker. Looking trough the mailing list archives I found out that this is "normal" for the modem and found settings that work in real usage.

Armed with that knowledge I got the default installation going nice (event the text-to-speech function work) and now I'm experimenting with the phoned-in-podcast idea with the sparse free time I have.

An annoyance is that it seems it's not possible to turn of the modem speaker even with m/vgetty post-init-chat, stuff does not work properly and sooner or later it gets re-enabled, I think that I'm going to take the modem apart and install a mechanical switch (so in case I need to debug something I still can hear all the lovely DTMF/data sounds and my own voice trough it).

Also I need to record new menu messages (the distribution ones have been recorded with too high sample rate for my modem [and while rate conversion is simple it also pitches down the sound]), for which I need a microphone (the one I thought I had lives no more...), I'll see if the USB headset we have at the office can be made to co-operate with my Ubuntu test box.

For people interested here is the (CLI PHP) script I made to test the various settings of pvftormd (not finalized in any way, almost certainly will need editing for your particular setup [for example, check the available settings or modes as the script refers to them with pvftormd -L]):

#!/usr/bin/php
<?php
echo "\n";
if (   !file_exists($argv[1])
    || !preg_match('/\.pvf$/i', $argv[1])
   )
{
    echo "Give valid .pvf file as argument\n\n";
    exit();
}

if (!$argv[2])
{
    $samplespeed = 8000;
}
else
{  
    $samplespeed = $argv[2];
}

$modes = array();
$modes['Digi'] = array(4,5);
$modes['Elsa'] = array(2,3,4);
$modes['ISDN4Linux'] = array(2,3,4,5,6);
$modes['Lucent'] = array(1,2,3,4,5);
$modes['MT_2834'] = array(4); 
$modes['MT_5634'] = array(4);
$modes['Rockwell'] = array(2,3,4,8);
$modes['UMC'] = array(4);
$modes['US_Robotics'] = array(1,4);
$modes['V253modem'] = array(2,4,5,6,7,8,9,12,13);
$modes['ZyXEL_1496'] = array(2,3,4);
$modes['ZyXEL_2864'] = array(2,3,4,81);
$modes['ZyXEL_Omni56K'] = array(4);

foreach($modes as $mainmode => $subs)
{
    foreach($subs as $submode)
    {
        echo "\n*** MODE: {$mainmode} {$submode}\n";
        //$convert = "pvfspeed -s {$samplespeed} {$argv[1]} | pvftormd {$mainmode} {$submode} >test.rmd";
        $convert = "pvftormd {$mainmode} {$submode} {$argv[1]} test.rmd";
        echo "Executing: {$convert}\n";
        passthru($convert, $stat1);
        if ($stat1)
        {
            // Non-zero return value means failure
            if (file_exists('test.rmd'))
            {
            system('rm test.rmd');
            }
            continue;
        }
        /*
        $audio = "rmdtopvf test.rmd | pvfspeed -s 8000 | pvftobasic > /dev/audio";
        echo "Executing: {$audio}\n";
        passthru($audio);
        */
        $modem = "vm play -s -v test.rmd";
        echo "Executing: {$modem}\n";
        passthru($modem, $stat2);
        system('rm test.rmd');
        sleep(1);
    }
}
echo "*** Done\n";

?>

It will be given a path to a .pvf file as argument, it will then convert that file for each setting combination and try to play it trough the modem speaker, see your vm log, even if no sound is heard if no error is reported in the log then those settings might work, write them down for further investigation.

It's mostly some VHSs I've since gotten on DVD, some duplicates (due to getting presents or special boxed sets) etc, notably one good CD-player is included. In case you live in Finland check out the list, (you may need to live near Helsinki though, otherwise the shipping is likely going to cost more than the actual items you want).

Tomorrow is training again, and this time we're training outside the stuff that requires a lot of space (which the inside of the HTC gym does not have), ie long weapons (even the sword is a long weapon inside the gym, there just isn't headroom...), I hope the weather stays good (at least the forecast is promising). In case you want to check us out we're somewhere around F-Secure office (Google maps) in Ruoholahti, but please do not disturb the training (there will be flyers and hopefully someone to explain to interested looking people who hang around).

First one was charset problem, the import did not check the charset of the incoming city name and thus LATIN-1 (sms) vs UTF-8 (server) lead to not finding locations with accented characters, also I imported the rest of the swedish locations from Geo-Names (Bergie had used some sort of population filter in his previous import).

Second one manifested as failed deletions but was in fact much more fundamental, the location report runs in midcom-exec, however Bergie had failed to 1. specify domain on request_sudo (required when in exec) 2. check the result (though I admit I also often forget to check the result of request_sudo). Fixed by specifying the domain and added some sanity checks.

Incidentally the Mail_Mime issue from yesterday, was found due to them as well.

I've also been busy as hell the whole spring and early summer, thus no meaningful updates, I didn't even remember to mention that I have most (but not yet all) of the photos I took in Taikai Finland 2006 up.

Then there's the "small matter" of getting a house built (which has been brewing for a while but now things need to start moving for real), but you'll hear more about that either when it's done (slightly over a year from now) or I feel too much like ranting about how difficult/hectic some things can get.

Mail_Mime

Today I found an interesting bug in Mail_Mime, mail_mimedecode specifically, if the MIME boundary contains single quote in the (in this case doublequote limited) value mail_mimedecode will thinks that is the closing quote and thus cannot properly resolve the boundary. The way we ran into this is that Epoc mail uses date somehow in generating the boundary it uses, yesterday things (send photo via email from phone, get in imported to the blog) worked, today two different phones using Epoc mail and emails sent from both started failing the import. For some reason today all mails sent from those contained a single quote in the boundary thus breaking the decode.

Due to time constraints I did not fix this in mail_mimedecode (but I need to remember mail a few pointers to the developers), in stead org.openpsa.mail I look for the boundary and check for the quote inside, if found I generate a new boundary (making sure it's not found inside the message) and do a replace.

Voice modem fun

<

p>I've also been playing around with vgetty and VOCP, though so far it has been mostly about vgetty and getting to the final realization that the modem I had laying around wouldn't cut it, so I got from the Finnish equivalent of ebay another one, when it arrives and provided I have any anough time to play with it there should be some interesting things coming (first thing we had in mind was to allow the boys to phone in podcasts from the road.

So a couple of phone calls to determine what to do next I was at the police station filing a report on lost license plate, and unless someone comes across it and returns it to the police in the next few days (not very likely as the likely place for it to have broken of is the Länsiväylä freeway, not a place to go looking for lost stuff smaller than A4 paper), I will have to go and get a completely new registration number (all plates for valid numbers must be accounted for) for the bike, which will cause all sorts of headaches since I have to inform the insurance company and various other parties about it.

Even more annoying I lost my hard earned Iron Butt Association license plate back.

In other news I'm looking forward to the Midgard developer meeting and also flashed my N770 to the OS2006, though haven't actually tested any of the features yet but it looks very good.

In other news I was two weeks ago at the Kaigosuru 2006, which was great (I'm really looking forward to seeing Arnaud again on Taikai Finland), unfortunately I didn't have much opportunities to take photographs (I wanted to train too...) but the ones I took can be seen on my home server.

The riding season is open, last thursday I got my bike back from the shop where they were installing side panniers, friday and saturday I made some shortish rides (It's cold and my weatherproof riding pants are undergoing maintenance [re-proofing etc]) then the weather got sour and I spent my easter playing God of War in stead. Anyways if weather is not intolerable I ride to work.

Linux tips & tricks

The main reason to write this entry is not to forget I nifty thing I finally figured how to do with NetCat, in debugging the new SyncML connector for OpenPSA 2 we need to see what the device and server are really sending each other, NetCat is the obvious choice but making a logging proxy with it is not as simple as one might think at first (the first time I tried to figure it out by myself I didn't succeed).

Anyways here is a shell script to set a logging proxy up (some ideas blatantly ripped from NetCat tutorial here (use tail -F inflow outflow [preferably in another window] to view the traffic after starting the proxy):

#!/bin/bash
FROMPORT=8081;
TOPORT=8080;
TOHOST=localhost;

mknod backpipe p
STATUS=0;
while [ $STATUS -eq 0 ];
do
    echo "$FROMPORT -> $TOHOST:$TOPORT"
    nc -l -p $FROMPORT 0<backpipe | tee -a inflow | nc $TOHOST $TOPORT | tee -a outflow 1>backpipe
    STATUS=$?;
done
rm backpipe

We use the mknod to create a FIFO we can use to pass data between the tees and NetCats (at first I also tried with pipes but soon realized that they are uni-directional and thus do not work for this), use ctrl-c to exit the loop (also happens if NetCat terminates abnormally), the FIFO is cleared then but the inflow and outflow logs are left in place.

Update, this can of course be adapted to SSL/TLS services as well using stunnel, the one below makes local plaintext socket and connects to remote SSL/TLS service (in the example case LDAP)

#!/bin/bash
FROMPORT=389;
TOPORT=636;
TOHOST=;

mknod backpipe p
STATUS=0;
while [ $STATUS -eq 0 ];
do
    echo "$FROMPORT -> $TOHOST:$TOPORT"
    nc -l -p $FROMPORT 0<backpipe | tee -a inflow | stunnel -c -r $TOHOST:$TOPORT | tee -a outflow 1>backpipe
    STATUS=$?;
done
rm backpipe

Converting this the other way around (listening on local secure socket and connecting to remote plaintext service) is left as an exercise to the reader (hint: use -d instead -c and switch the command places, then you still need to configure the certificate, see stunnel manual)

Electronics

I've also been playing around with this GPS datalogger on my spare time, trying to get it to work with any serial NMEA source, so far the "easy ways" have been shown not to work, next I need to try a MAX232/233 based solution for changing the signalling from RS232 to 3.3V TTL (I sure hope I have not friend the corresponding input pins on the controller with my earlier experiments but I live in the belief that if something would be fried it would be the whole uC).

I also have some ideas on improving the firmware of the same (check it out, it's dead simple, even for my very limited "may understand what he reads" C skills), but first I need to figure out how to compile (this should be doable with GNU gcc) my changes and then flash (in circuit, this is the hard part) them onto the chip, without paying trough the nose for "professional" tools.

The "deluxe thermostat" is a circuit that controls a relay, it keep the relay engaged until a set temperature (controlled with a trimmer on the board) is reached and then allows the temperature to drop until another set (controlled with another trimmer) is reached, then engages the relay again until the higher temperature is reached. lather, rinse, repeat.

The circuit works just fine, however the fast-setting epoxy I used to glue the sensor to the wall of the samovar apparently isn't designed to handle temperatures over 80C and when the water got to boiling the epoxy became goo not suitable for holding a sensor in place, so I had to scrape it off and before we can actually use it and tune the temperatures I need to find better epoxy.

Meanwhile enjoy these (low quality) pictures from my "lab" (the circuit manufacturing side, computer, printer, board cutter, development solution, exposure unit, plain water and acid). Not seen (behind the camera), is the "drilling station" (dremel with a vertical mount) and "soldeting/prototyping station" (just a table with mess of parts, function generator, ancient oscilloscope and a prototping board, I use Weller Pyropen Piezo for soldering work).

But for the benefit of the world find the circuit attahced, in Eagle (a nice electronics CAD package, the free version is more than enough for most hobbyists) format. The temperature sensor (LM35DZ) I used was not available in the shipping libraries and I was too lazy to create it as component so I used another sensor, but it had different pin assignments, thus the weird connections. the smaller MOSFET can be just about any N-type one, same should go for the power transistor used with the relay.

Power the circuit with whatever DC your relay requires, as long it's over +5V, I used Kawamisawa(?) JS-12-N relay, which runs on 12V/1A (and can easily handle 10A of 220V AC) and the power supply is 12V/1.5A generic supply.

• Sahara, not unexpectedly was not that deep on the plot end, however it was very well executed and immensely entertaining adventure/action movie.
• Danny the Dog, while the main plot (emotionally FUBARred guy meets blind piano tuner and his pianist daughter and becomes a human being again) is cliche to say the best the movies is still really good, I especially liked the editing combined with fabulous music by Massive Attack
• Coronado had less action and more adventure than Sahara, but was not as much fun. Quite entertaining but nothing out of the ordinary. Nice settings though.
• Bugi Fiction, not Finlands greatest gift to the world of movies, a small budget independent parody production. It has it's moments but... Be drunk (I couldn't be, though I still finished the movie).

Recently I finished watching Babylon 5 (there is a story involved with this as well but I'll write it down another day) again, and on sunday evening when Taru got back we decided to check out the Crusade, now I've heard it doesn't exactly measure up to the predecessor, but based on the pilot episode that is understatement of the year. We'll check out a few more episodes to decide whether we try to endure it any further.

So I needed to upgrade just the database, preferably so that I don't have to rebuild each package that uses the MySQL client library again.

After more than should be neccessary searching I found this article on building MysQL4 for FC3, it involved a bit older versions of the source packages and there were some other issues as well leading me to recompile the package (no small operation even on quite fast machine) numerous times before getting all the kinks out, but finally I got it to build cleanly.

The fc3_mysql4.tar.gz file contains the binaries I built and my versions of the SPEC files, tested to work with out of the box Midgard 1.7.4 from the Midgard YUM repository.

It started last sunday (in retrospect attending the mini-seminar on saturday might not have been such a good idea), and while I could do stuff on computer then by monday morning I was bedridden.

The next few days were spent quite firmly in bed (apart from a visit to the doctors), too tired to even write about my past exploits (in general being sick at home offers a good time for such reflections)

Today I came to the office to check the snailmail and handle some urgent matters, now that they're done I think I'll head back to bed so I have more time to recover for monday.