Extracting SMS Text Messages from a Palm Treo

SMS Messages on the Palm Treo PDA

Like most SMS ("text messaging") capable phones, the Treo saves in memory all messages received or sent. I got to wondering if there was a way to extract all of the saved messages for searching and archiving.

Since the Treo is of course a palm, those messages are saved in a palm-format database, which can be synced to a desktop computer. The SMS messages are saved in a database called "Messages Database.pdb"

This page is about finding the format of that database, and extracting the information.

Other Tools

The message database (and other databases) can be copied to a desktop PC using Pilot-Link

The message database contains multiple records, usually one for each text message. The records can be extracted into individual files, using the Par palm archiver tool.

Extract all of the message records with a command like "par x 'Messages Database.pdb'"

The Mesage Database Record Format

While the palm database format allows tools like Par to find each record, the format of the records is undocumented.

Hints about the record format can be obtained by studying the program sms2csv.php. Thanks very much to Livy for making that program and its source available.

I'd prefer a batch-mode program, running on my desktop - so armed with sms2csv an a hex-dump utility, I set out to write a C version.

The message record format

Each message record appears to consist of a series of blocks. Each block starts with a 16-bit block type. Some block types are fixed length, for example 0x0007, which contains a 4-byte integer representing a date and time. Others are variable-length, and consist of a 16-bit integer containing the length of the block, followed by that many bytes of data.

Single-message converter: dumpmsg

The first useful result is dumpmsg.c, a program to take a file containing a single message-record (as extracted by Messages Database.pdb using Par) and format the message in a convienient manner.

Example

This example shows running dumpmsg on one of the files resulting from running "par x" on Messages Database.pdb:

$ dumpmsg 270.40.14766972.pdr 
From 15002345678 Sat Apr  5 10:09:59 2008
From: "My Friend (M)" <15002345678>
Date: Sat Apr  5 10:09:59 2008
X-TreoTag: Trsm
X-InputFileName: 270.40.14766972.pdr

Hooray!

This shows that this message came from my friend at phone number 15002345678 (not a real phone number) on Saturday April 5, and contained the text "Hooray!". The output is close enough to the old standard unix "mbox" format that some email programs can read it and treat each SMS message as if it were an email message.

Running on another message produces:

$ dumpmsg 380.40.14767146.pdr 
From unknown Sat Dec 31 00:00:00 1969
Content-Type: image/jpeg
X-Filename: Photo_061708_001.jpg
X-InputFileName: 380.40.14767146.pdr

(nontext message body of 55946 bytes)

This message contains a jpeg image instead of plain text. We can extract it with a command like:

$ dumpmsg -o myphoto.jpg 380.40.14767146.pdr

Compiling the program

On Linux, download the tarfile, unpack it, and run "make". On other operating systems, you're on your own.

Future Work

Several block types in the message records aren't fully understood; with further study they may be supported better. The "-x" option to dumpmsg dumps the block structure of a message as currently parsed by dumpmsg; sometimes studying it can help improve the program.

Eventually it would be nice to write a new program incorporating code from dumpmsg and code for parsing the generic palm database format, so that the mult-step process of running Par followed by dumpmsg on each file can be simplified to a single step.

Author

This page, and the dumpmsg.c program, were written by Stephen Tell.