I've been mulling the idea of keeping my lab notes on my Mac for a while, and I've started moving in that direction. The problem with keeping notes on a computer rather than on paper is that the computer is far less flexible. It's much more powerful, but it's quite constrained by the need for exactly the right software. The major advantage of a computer over a lab notebook is that you can put in a whole lot more data, and interlink the data in ways that you just can't with paper.

The ideal lab notebook software would combine some of the functionality of a blog with some of the functionality of a wiki. The blog function would be to simply keep a log of all entries, with timestamps. The entries would consist of text, images, and tables of data. The wiki function would integrate the linear collection of entries from the blog to build up a coherent time-independent picture of the object under study. The wiki would include both information about the current state of the experiment and a set of tentative conclusions about the phenomenon under study, along with things like lists of references with comments.

As an example of how such a system might work, let me use my recent lab work. We blew up a voltage divider used to measure high voltages last week. In the blog notes I would note that the voltage divider is kaput and how it got that way, and crosslink to the wiki entry showing the current status of the machine, updating it to show that the voltage divider is now gone, with a linkback to the blog entry. Now I need to design a new voltage divider, so I go to the blog function and lay out a list of requirements for the new voltage divider, and begin designing. Right there in the blog I carry out the calculations using a calculator plugin which logs the calculations with annotations. Having picked resistor values I now go build the thing, based on rough sketches done directly in the dynamic lab notebook. When the new divider is finished I take a picture and upload it to the blog, with markups and annotations as needed. Then I install the divider in its new location, take another picture and upload it. The final step is to update the wiki section to reflect the new voltage divider parameters and link to the blog entries describing the design and installation process.

Following the procedure above it's relatively straightforward to keep an up to date set of documentation on the machine, and if there is a problem the blog entries can help unravel it. The big issue is keeping the overhead low. If it's a PITA to use the system it won't get used. Right now the long pole in the tent seems to me to be loading pictures, which currently takes about 15-30 seconds all told (including going to the machine, plugging in, downloading and dropping into place). Doing that 10 - 15 times a day adds enough overhead to be a problem. It would be nice if there was a way to simply fire the pictures wirelessly to the blog in real time so they are saved, with timestamps. The other problems are software (and the picture problem might be software fixable, given the right camera).

I've installed Twiki on my machine and I'm starting to figure out how to configure it to do what I want. I chose Twiki because it's widely deployed and has a lot of plugins, with more in development. The ideal plugins for my purpose don't yet exist, but it looks like they might be adapted from some of the existing Twiki plugins. In particular the really important ones are something to do simple photo markup (crop, add arrows, add text, perhaps draw freehand), a calculator (perhaps something derived from Frink), a simple sketch program with enhancements for including measurements (IOW, no need for any color effects, gradients, etc. just easy line drawings), and a way to directly edit pages in a semi WYSIWYG fashion (this is in development for Twiki). I think most of the rest of the functionality needed either already exists or is relatively easy to implement.

The ideal longer term plan would be to implement a system where LabView can be set up to automatically generate a link on the blog to data as it is taken, so that the blog tracks absolutely everything.

Anyway, I'm still in the early stages of exploring this idea and trying to see if I can actually implement it and make it useful. I figured I'd blog it to see if anyone has interesting comments or insights that might make it a little easier.

Interesting idea. Are you considering integrating this into a TabletPC or PocketPC? This might make a more convenient "drop-in" replacement for a paper notebook. It would have to be something that can be easily carried around a lab, and fit easily on a cluttered lab bench, and be fairly durable. Most notebooks probably won't fit these requirements.

Also, maybe because I'm in engineering, most of my notebooks are filled with sketches, diagrams, and flowcharts, along with numerical datapoints. So you would have to have some sort of easy-to-use drawing program. Perhaps the stylus-on-screen interface of a tabletPC would be best. I don't think that CAD software would be appropriate. Maybe I'm old-fashioned, but I usually sketch designs with pencil and paper before I put I use CAD. It's just more intuitive.

Finally, the LabView interface is a great idea. Most modern instruments, even scales, are microprocessor controlled. Ethernet or bluetooth interfaces are inexpensive relative to the cost of a good instrument. Why not have a button that can just automatically transmit a measurement to your electronic notebook? It would save time, and increase accuracy (I have terrible handwriting, and sometimes even I can't read what I wrote). I guess this becomes larger than just notebook software, though.

One outstanding issue would be the status of a lab notebook as a legal document. Theoretically, we are supposed to sign and date our notebooks everyday, and have possibly important inventions or discoveries signed by a witness. This is in case notebooks have to be produced as evidence in a intellectual property trial. I know of no one who signs their notebook everyday. But I have witnessed other books, and have had people sign off as witnesses on mine. I don't know how this would apply to electronic records.

Neat idea! Let me know when I can try out your software! :)

I second the concern about 'labbook as a legal document'. I have a pile of things on the computer, and several logbooks and labbooks. Moving the thoughts and ideas I express in my books onto my Mac would be great - but the timestamps are all completely fungible -> hard to pin down _proof_ of when this thought happened, or when you changed methods to method X etc.

Good point about timestamps and legal issues. I think this has been handled already in other contexts, but it's worth bearing in mind. There are probably ways to generate a hash function and get a third party to log it so that you have evidence a particular entry was made at a particular time without exposing the contents. I wonder if there's a business in that.

As far as Tablet PC - I really like having a keyboard. I'd much rather work through the keyboard, but for sketching a tablet is really good. There are hybrid tablet/laptop machines out there, so maybe that's the way to go. One thing that would be nice for me since I occasionally do really dirty work, is a disposable keyboard cover - something like disposable gloves, but which fits over the keyboard.

Another niche the Newton could have filled if it lived. I wrote some Newton programs for field use that worked great.

The problem with a portable computer is that you can't use it standing up while holding it, and the screen geometry is wrong. You need a tablet with pen of some sort with a detachable keyboard for heavy text entry. Also, you might want to make sure it can handle the conditions, especially if like me, you use it outdoors in adverse weather.

If conditions are an issue, innstead of going with a all-in-one package, you might consider a wireless "terminal" to a real computer. That way when the terminal dies because you got caught in a sudden downpour (speaking from experience) you won't lose data. (And set up that machine for frequent backups, too.)

But there's one drawback that kept me from commiting to the Newton completely-- there's something about being able to add a completed notebook to the shelf you can't do with a computer.

I personally enjoy taking lab notes on my Treo 600. It fits in your pocket, and the thumb keyboard is mighty useful.

It might also be interesting to use something like a Treo to edit a wiki over wireless. Latencies might be annoying, though.

Neil - if you are ever in the DC area drop a line. It looks like your research overlaps considerably with my wife's research interests (neuroscience, specifically cognition). If nothing else I can introduce you to some people who might be good to know.

here ya go: http://www.dpreview.com/reviews/nikond2h/page15.asp

quote:
Nikon D2H is in the unique position of having an optional add-on (the WT-1) which enables WiFi (802.11b) transmission of images to either a PC with WiFi capability (ad-hoc connection) or a Wireless Base Station (infrastructure connection). Transmission is carried out over standard TCP/IP using the FTP protocol

once it's on the server, a bit of applescript will do whatever you need done with it.

also, have you seen Devonthink? http://www.devon-technologies.com/products/devonthink/overview.php

Jay - thanks for the pointer. I've seen Devonthink, but rejected it due to the proprietary data format and the hassles with licensing. It's close to what I want, but I don't think its as extensible as I'd like.

The Author, you - genius...
[url=http://taxi-crimea.com]taxi[/url] http://taxi-crimea.com
All the best!