Org-mode, documents, and XDS

I've created an org-mode date-tree for organizing my various documents. This leads to some thoughts on XDS, etc.

Org-mode stuff (skip this if you don't care much about org-mode)

• I've created a date-tree for these documents. 
• Each set of documents gets a headline, with whatever description, tags, and metadata I feel like adding.  This date-tree lives in my regular org files.  So I can now search for tags, concepts, dates, etc. to find headlines.  Then I can read the headlines, their descriptions, and their metadata in order to decide which documents I want to read.
• The attachments are in their own git archive.  I can make this work by having a different attachment directory in each of the .emacs files on each machine.  My first thought of using links in the file system failed because Windows XP does not support links.
• I still need to do extra manual work, but it's now tolerable.  I've created a temporary save area.  I detach the documents from an email into that save area.  I create a headline in the date-tree. The headline text includes key aspects from the email, if any, and a short description of the documents.  I then attach the documents from the temporary save area (which copies them into the org attachment directory) to the headline.  Finally, from time to time I empty the save area.
• The org attachments directory is outside the org tree so that my git sharing of org files doesn't pick up all the attachments too.  Those are managed separately, also by git sharing.  This lets me do things like put the attachments directory on a removable drive.  (This makes the org file vaguely like the XDS Registry, and the attachments kind of like an XDS Repository.)
• Attachments management is adjustable by machine.  On the Mac the nicest way to deal with them is to open the attachments directory for an entry, and then use Mac tools.  On the Linux and Windows, the easiest is to do directory management within emacs and open files from there.  (XDS folks can think of the attachments for a headline as being like a submission set.)
• I had to fix the ~/.mailcap on my Linux system to point the PDF relationship at "evince".  Emacs was opening the PDF files within an emacs buffer.  This works, but it's slow and has a lot fewer features.  It works by converting the PDF into a PNG and displaying that with image display tools.  That's not as good as a full function PDF viewer.  The Mac and PC already have PDFs pointed to the appropriate viewer.

Resulting thoughts on organizers and XDS

This raises some thoughts on the state of information organizers.  Most people still use something like Sharepoint with all of it's preconfiguration restrictions and rules.  Some use wikis, which don't have the tags, properties, and other metadata facilities of org.  (Most wikis also have a lot more problems with editing the content.  The good content editors are platform specific.)  There are various more sophisticated metadata management tools that do a better job with document management.

Org still has too many manual operations.  Properties, tags, etc. are good, but don't integrate well with ontology tools.  (On the other hand, org has no problems with multiple ontologies.)  Tags, etc. are at the level of headlines, so there can be inheritence and the like.  But this is all stuff that you need to set up for yourself.  So it's a tool for tool builders, not a tool for general users.

The potential is there to do all kinds of things since this is build upon a structurally aware lisp engine.  I think I could make this into an integrated XDS Registry/Repository if I felt like it.  The really hard part would be implementing all the SOAP crap and enforcing metadata rules.  Org's attitude toward metadata is "do whatever you want." Org tries to adapt as best it can.

My org approach doesn't have the XDS single registry viewpoint.  But that's because I'm accustomed to systems  and system designs that are normally "broken".  They are inconsistent in contents, etc. in that each different viewpoint may reflect a different set of updates and changes.  The processes exist to reconcile the changes, but there is no requirement to present a consistent or "accurate" viewpoint.

This kind of "it depends on where and when you look" is an inherent necessity for many operational environments.  Presenting a single view requires 100% connectivity, 100% functional, error free network and systems.  In some enterprise environments it is practical to achieve this, or close enough that people don't notice the failures. 

In telecoms management this is impossible.  Networks will have failures that partition the network.  The various parts must continue to operate and be managed based on what information is still available, and they must adapt to mergers, including reconciliation of conflicting information when merging.  In the largest of the networks that I was involved with we never had fully functional networks.  There was always a portion of the network that had failed.  There were always a few islands operating on their own while problems were being fixed.

Healthcare IT folks (like those involved with XDS) are still having real problems coming to grips with accepting a system design that is always "broken".  They don't realize that in healthcare  everything is always "broken" in that sense.  Patients don't provide full or consistent  information, so patient information is "broken".  Measurements are incomplete.  Diagnoses are really just working hypotheses to be tested.  Disease and health are constantly changing, so the recorded state is never correct.  It's like large networks.  You must proceed with incomplete, inconsistent, changing information.

With time they should come to grips with this.