tag:blogger.com,1999:blog-53098135651859180442024-03-14T01:13:13.383+01:00Dust FeedNiklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.comBlogger45125tag:blogger.com,1999:blog-5309813565185918044.post-22898856234350006942011-11-09T19:14:00.003+01:002011-11-09T19:20:33.728+01:00One Liner At a TimeJust some one-liners. 'Cause, you know, to post something, and to each one its own.<br /><br /><pre><code>even_map = dict(('key_%s' % i, i) for i in range(1, 9) if i % 2 == 0)</code></pre><br /><pre><code>even_map = Hash[(1..9).find_all { |i| (i).even? }.collect {<br /> |i| ["key_#{i}", i] }]</code></pre><br /><pre><code>(def even-map (into {} (for [i (range 1 9) :when (= (mod i 2) 0)]<br /> [(keyword (str "key_" i)) i])))</code></pre>Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com0tag:blogger.com,1999:blog-5309813565185918044.post-121335401038234122011-07-03T12:30:00.008+02:002011-07-03T17:50:15.982+02:00Resources In Various Frames of JSON<div>I've been mulling over the role(s) JSON should play in representing RDF the last couple of days (well, the last year or so really).</div><div><br /></div><div>Having worked with RDF for some years now, more or less full time, in specific contexts (legal information, and lately educational data), I'm getting a hang of some usage patterns. I'm also following the general Linked Data community (as well as various REST endeavors, Atom applications, and various, mostly dynamic language based, platforms). In all of this, I've made some observations:</div><div><br /></div><div><b>1.</b> If I want to use RDF <i>fully</i>, anything but a full RDF API (Graph navigation, Resource and Literal objects and all) often cause a lot of friction. That is, unless I already know what "framing" I need of the data, I really need a whole data set to dig through, for purposes of e.g. finding incoming relations, looking up rdfs:label of types and properties, filtering on languages, irregular data (such as dc:creator used both with literals or URI references) and so on.</div><div><br /></div><div><b>2.</b> Encountering general JSON data (not RDF specific) on the web, I sometimes come across quite crude stuff, mostly representing a database slice, or gratuitous exports from some form of O/R mapper. It <i>may</i> look meaningful, but often shows signs of ad-hoc design, unstable modeling without "proper" domain understanding and/or implementation leakage. <i>However</i>, the data is accessible for most web programmers without the need to get the domain properly, no matter how poor this data representation may be. JSON is language native. The use case is to have users (programmers) be able to spin around a specific, boxed and digested slice of data. Ideally you should also be able to find and follow <i>links</i> in it. (Basically the values which match something like /^(http(s)?:|\/|\.\/)\S+/ ...).</div><div><br /></div><div><b>3.</b> If I know and control the data, I can frame it in a <i>usage scenario</i> (such as for rendering navigable web pages with summaries of entities) based on a specific domain (such as an article document, its revisions and author, etc.). Here is great potential for <i>reducing</i> the full data into a something, raw, more (web) programming language native. This is where a JSONic approach fits the bill. Examples of how such data can look includes the JSON data of e.g. <a href="http://data.nytimes.com/60694995023816375851.json">New York Times</a>, <a href="http://id.loc.gov/authorities/sh95000541.json">LCSH</a>. The <a href="http://code.google.com/p/linked-data-api/wiki/API_Formatting_Graphs">Linked Data API JSON</a> is especially worth mentioning, since they explicitly reduce the data for this <i>casual</i> use so many need.</div><div><br /></div><div>Point <b>1</b> is just a basic observation: for general processing, RDF is best used (produced and consumed) as RDF, and nothing else. It can represent a domain in high fidelity, and merges and is navigable in a way no other data model I've seen supports.</div><div><br /></div><div>Point <b>2 </b>is about quick and sometimes dirty. Cutting some corners to get from A to B without stopping for directions. You cannot do much more than that though, and in some cases, "B" might not be where you want to go. But it works, and if the <i>use case</i> is well understood, anything more will be considered waste for anyone not wanting the bigger contexts.</div><div><br /></div><div>Point <b>3</b><i> </i>then, is about how to go from 1 into 2. This is what I firmly believe the focus of RDF as JSON should be. And since 2 is <i>many things</i>, there may be no general answer. But there <i>is</i> at least one: how to represent a linked resource on the web, for which RDF exists, as a concise bounded description, showing inherent properties, outgoing links and <i>per application considered relevant</i> incoming links. And how to do this in JSON in <i>high fidelity</i> but immediately consumable by someone not wanting more than "just the data".</div><div><br /></div><div><div>Many people have expressed opinions about these things of course. You should read posts by e.g. <a href="http://www.ldodds.com/blog/2010/12/rdf-and-json-a-clash-of-model-and-syntax/">Leigh Dodds</a> and <a href="http://webr3.org/blog/linked-data/opening-linked-data/">Nathan Rixham</a>, and look at some <a href="http://www.w3.org/2011/rdf-wg/wiki/JSON-Serialization-Examples">JSON Serialization Examples</a>. Also monitor e.g. the <a href="http://lists.w3.org/Archives/Public/public-linked-json/">Linked JSON W3C mailing list</a> and of course the ongoing work of <a href="http://json-ld.org/">JSON-LD</a>. Related to the Linked Data API and its "instrumental" JSON is also a recent presentation by <a href="http://www.jenitennison.com/blog/">Jeni Tennison</a>: <a href="http://www.slideshare.net/JeniT/data-all-the-way-down">Data All the Way Down</a>. It's short and very insightful. <i>End-users</i> have different needs than <i>re-users</i>!</div><div><br /></div></div><div>Early on (over a year ago) I drafted <a href="http://code.google.com/p/oort/wiki/Gluon">Gluon</a>. I have not used that much since. A related invention I <i>have</i> used though, is <a href="http://code.google.com/p/oort/wiki/SparqlTree">SparqlTree</a>. While it isn't really a mechanism for defining how to map RDF terms to JSON (but to formulate SPARQL selects digestible into compact results), it does so quite well for specific scenarios. It is very useful to create <a href="http://manu.sporny.org/2011/semweb-problems-1/">frames</a> to work on, where code paths are fully deterministic, and where there is a one-way direction of relations (which is needed in JSON trees, as opposed to RDF graphs where we can follow rel and rev alike). Admittedly I've done less than I should to market SparqlTree. But then again, it is a very simple and instrumental solution over an existing technology. I recently gave a glimpse of how I use it in a <a href="http://lists.w3.org/Archives/Public/public-rdf-dawg-comments/2011Jun/0014">mail concerning the "Construct Where of SPARQL 1.1"</a><a href="http://lists.w3.org/Archives/Public/public-rdf-dawg-comments/2011Jun/0014"> </a>.</div><div><br /></div><div>Reflecting on all of this, I'm quite convinced that anything like RDF/XML or Turtle is beyond what JSON should ever be used for. That is, support for all kinds of general RDF, using prefixes (whom I <i>love</i> when I need to say anything about anything) and exposing the full, rich, internationalized and richly and extensibly datatyped world of literals is <i>beyond</i> the scenarios where JSON is useful. If you need full RDF, use Turtle. Seriously. It's the best! It's rather enjoyable to write, and I can consume it with any RDF API or SPARQL.</div><div><br /></div><div>The only case I can think of where "full RDF in JSON" might apply is for machine-to-machine data where for some reason only JSON is viable. For this, I can see the value of having Talis' <a href="http://docs.api.talis.com/platform-api/output-types/rdf-json">RDF/JSON</a> standardized. It is used in the wild. It is reminiscent of the SPARQL results in JSON, which for me is also quite machine-like (and the very reason for me inventing SparqlTree in the first place!). I'd never hand-author it or prefer to work on it undigested. But that's ok. If handed to me I'd read it into a graph as quickly as possible, and that'd be dead simple to do.</div><div><br /></div><div>So where does this leave us? Well, the Gluon I designed contains a general indecision, the split into a <i>raw</i> and a <i>compact</i> form. The problem is that they are overlapping. You can fold in <i>parts</i> of data into compact form. This is complex, confusing and practically useless. Also, the raw form is just another one in the plethora or more or less "turtle in JSON" designs which cropped up in the last years. I doubt that any such hybrid is usable: either you know RDF and should use Turtle, or you don't and you want <i>simple</i> JSON, without the richness of inlined RDF details.</div><div><br /></div><div>My current intent is to remove the raw form <i>entirely</i>, and design the profile mechanism so that it is "air tight". I also want to make it as compact as possible, true to RDF idioms but still "just JSON". A goal will still also be that <i>if present</i>, a profile should be possible to use to get RDF from the JSON. This way, there is a possibility of adding the richer context and integratability of RDF to certain forms of well designed JSON. This of course implies that Gluon-profile compatible JSON will be considered well designed. But that is a goal. It has to <i>look good</i> for someone not knowing RDF!</div><div><br /></div><div>I have a strawman of a "next generation gluon profile"<a href="http://code.google.com/p/oort/source/browse/etc/gluon/drafts/profile-redesign.json"> in the works</a>. I doubt that you can glimpse my design from that alone, but anyway.</div><div><br /></div><div>Some things to note:</div><div><ul><li>The 'default' feature will be more aligned with the @vocab mechanism of RDFa 1.1 (and JSON-LD)</li><li>Keywords ('reserved') can be redefined. There are preset top-level keys, but that's it. (A parser could parameterize that too of course.)</li><li>No CURIEs - every token is "imported" from a vocabulary.</li><li>Types will be powerful. They'll determine default 'vocab' for a resource description (i.e. JSON object), and you can also import terms locally for a type (so that a Person title is foaf:title although 'title' is globally from 'dc').</li><li>If there are multiple values for a term (i.e. multiple triples with the same subject and predicate), a defined prefix or suffix will be added to the term. This is an experiment to make this nagging problem both explicit and automatic.</li><li>The 'define' will be reduced to a much less needed component. Using 'autocoerce', pattern matching on values will be bravely used to coerce mainly date, dateTime and URI references to their proper types.</li><li>Incoming links can be represented as 'inverseOf' attributes, thus making it possible to frame more of a graph as a tree.</li><li>Named bnodes are out (though they <i>might</i> be snuck in via a "_:" link protocol..). Anonymous bnodes are just fine.</li></ul><div>This is a design sketch though. Next steps are to work on adapting my test implementations and usage thereof.</div><div><br /></div><div>An auxiliary but very interesting goal is the possibility of using these profiles in a high-level API wrapper around an RDF graph, making access to it look similar to using Gluon JSON as is (but with the added bonus of "reaching down" the abstraction to get at the details when needed). (This is the direction I've had in mind for any development of my nowadays aged <a href="http://oort.to/">Oort</a> Python O/R mapper. More importantly, the current W3C RDF/Structured Data API design work also leans towards such features, with the Projection interface.)</div><div><br /></div><div>(Note that profiles will reasonably not be anything like full "JSON schemas". It's about mapping terms to URI:s and as little else as possible to handle datatyping and the mismatch between graphs and JSON trees. There is a need for determining if a term has one or many values, but as noted I'm working on making that as automatic as possible. Casting datatypes is also needed in come cases but should be kept to a minimum.)</div></div><div><br /></div><div>Finally, I really want to stress that I want to support the progress of JSON-LD! I really hope for an outcome to be a unification of all these efforts. The current jungle of slightly incompatible "RDF as JSON"s sketches is quite confusing (and I know, Gluon is one of the trees in that jungle). I believe JSON-LD and the corresponding W3C list is where the action is. Since there is work in JSON-LD on profiles/contexts, and a general discussion of what the use cases are, I hope that this post and my future Gluon profile work can help in the progress of this! For me Gluon is the journey and I hope JSON-LD is the destination. But there are many wills at work here, so let's see how it all evolves.</div><div><br /></div>Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com0tag:blogger.com,1999:blog-5309813565185918044.post-48368547055214437952009-12-31T00:03:00.002+01:002009-12-31T00:09:51.170+01:00The Stone in the House of Glass<div>The stone in the house of glass</div><div>began to tumble</div><div>It didn't really see</div><div>where it was going</div><div><br /></div><div>The stone in the house of glass</div><div>began to rumble</div><div>It didn't really hear</div><div>what it was doing</div><div><br /></div><div>The stone in the house of glass</div><div>took a dive</div><div>It didn't really sense</div><div>its own surroundings</div><div><br /></div><div>The stone in the house of glass</div><div>began to crumble</div><div>It didn't really know</div><div>its limitations</div><div><br /></div><div>The stone in the heap of shards</div><div>has smashed the building</div><div>It didn't really get</div><div>what it was all about</div><div><br /></div>Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com2tag:blogger.com,1999:blog-5309813565185918044.post-54078731985793895702009-11-12T22:58:00.012+01:002009-11-13T16:07:34.470+01:00The Groovy Times<p>It has been so long since my last post here. I've <a href="http://twitter.com/niklasl">twittered</a> away like the rest of my peers. I guess I could dish out details from my personal life of the past year now. To examine my interrupt. I won't.</p><p>I've been using <a href="http://groovy.codehaus.org/">Groovy</a> a lot in my work on the <a href="http://rinfoprojektet.wordpress.com/">Swedish Legal Information System</a>. While the <a href="http://semanticweb.org/">things</a> I find <a href="http://code.google.com/p/court/">interesting</a> in this work deserve <em>many</em> separate posts, I'll just spend this one to drop some nice stuff about groovy.</p><p>"Why Groovy", you might ask? Oh dear. For "political" reasons (this is an entire topic of its own), I have to use Java. But the language Java is often so much overwork and ceremony; riddled with convoluted ways to make explicit patterns and formalisms. Dynamic languages are pragmatic. Sure they have flaws, but I find the compromise acceptable. There are probably thousands of articles discussing this, and I prefer to debate it elsewhere (mostly with friends over lunch/dinner/beer).</p><p>I must deliver the bulk in Java, and Groovy is <em>impressively</em> close to Java, but with so much more expressive power (ease of use). (More than necessary? My pythonic side says "probably".) I've worked <em>a lot</em> with Jython the last decade, and some with JRuby. Groovy trumps them both (IMHO) when it comes to java library and java culture interoperability. I can spike, explore, and make tests in groovy. Then I add all this horrendous checked exception handling, spinkle some semicolons for the grumpy old compiler, and finally explode-a-pop all the <code>def</code>:s to bulky types when things need to "harden" (to fulfil the contract of delivering <code>.java</code>...; the tests are left in groovy). Not a big deal (especially since I use an <a href="http://www.vim.org/">editor</a> that makes code <em>editing</em> a breeze). And groovy cross-compiles nicely with traditional cruft.</p><p>So what have I used more specifically? <a href="http://code.google.com/p/spock/">Spock</a>. Check it out. Rewrite ten of your JUnit4 tests in it, and if you go back, write ten more. Don't go back. If you're already testing with say JRuby, I won't push you, but I assure you Spock is worth looking at. Specs become liberatingly clear and thin. Data-driving some of them is pure joy. Mocking is dead simple. (Sorry, I won't put code examples here now: look at <a href="http://code.google.com/p/spock/wiki/SpockBasics">the spock docs</a> for that. <em>Try them out</em>!)</p><p>For building, we do <em>not</em> use <a href="http://www.gradle.org/">Gradle</a> (not yet at least), but that scary beast of mindnumbing declarativity (which I'm usually for), dreadful xml (which I can handle due to prolonged exposure) and conflated purposes (no excuse here) known as Maven 2. It seemed paramount in the surroundings when we started, and won't go away soon. I use it as little as possible (and it <em>is</em> quite impressive when you let it do its thing).</p><p>Which leads me to the last thing I want to mention: how to use Groovy's very convenient, builtin <a href="http://groovy.codehaus.org/Grape">Grape</a> system (and its <code>@Grab</code> mechanism) together with my local maven2 artifact repo. That one in <code><~/.m2></code>, where all my local packages have been <code>mvn install</code>:ed (along with the umpteen dependencies).</p><p>The thing was, when I started, I naively thought things would kind of work at least semi-automagically. I've spent my time in <code>CLASSPATH</code> hell. I wanted groovy to tap into the local m2. Then I was disillusioned again, and attempted to run experimental groovy scripts via GMaven. Didn't fit my use cases at all (that thing is great at compiling, I leave it at that). I shellscripted the path from <code>mvn dependency:build-classpath</code>, then built pathing jars, then just felt quite uneasy (such moves work, but it's not particularly clean).</p><p>When Grape appeared I tinkered with the <a href="http://ant.apache.org/ivy/">Ivy</a> config in <code><~/.groovy/grapeConfig.xml></code>. It surely looks so simple. I couldn't figure it out. <a href="http://www.jroller.com/berni/entry/first_experience_with_grape">Benhard could.</a> Neat.. But alas, that solution <em>copies</em> all the dependencies from the local m2 repo to grape's ivy repo. And I could not get it to grab my new local SNAPSHOT-stuff as they landed either (in spite of eleventy ivy attributes claiming to force all kinds of checks).</p><p>Then it appeared, from a combination of fatigue and taking a step back (cue magic "aahh").</p><h4>Use Groovy's Grape with your Local Maven File Repo</h4><p>Locate <code>$HOME/.groovy/grapeConfig.xml</code>. If it doesn't exist, see the <a href="http://groovy.codehaus.org/Grape">Grape docs</a> for how the default version should look.<br /></p><p>Then add, directly after (xpath) <code>ivysettings/settings</code>, the following directive:</p><pre><code> <caches useOrigin="true"/></code></pre><p>And, in (xpath) <code>/ivysettings/resolvers/chain</code>, after the first <code>filesystem</code>, add:</p><pre><code><filesystem name="local-maven2" m2compatible="true"><br /> <ivy pattern="${user.home}/.m2/repository/[organisation]/[module]/[revision]/[module]-[revision].pom"/><br /> <artifact pattern="${user.home}/.m2/repository/[organisation]/[module]/[revision]/[artifact]-[revision](-[classifier]).[ext]"/><br /></filesystem><br /></code></pre><p>With that in place (pardon the line width), Grape (and thus <code>@Grab</code>) will happily use anything it finds in your local m2 file repo, without copying the jar:s. (It will still download other stuff to use to the default <code>~/.groovy/grapes/</code>, which is fine by me.)</p><p>That's it for now. There are lot's of cool stuff with Groovy, if you're in a Java environment and want to ackowledge that without giving up modern power.</p><p>Three years ago I looked at <a href="http://dustfeed.blogspot.com/2006/11/scala-genau.html">Scala</a> for the first time, and found it quite interesting. Then I got a bit spooked by the academic machinations of it. Lately that interest has been quite rekindled though, and the future will show what will come of that. I am very happy to have used Groovy so far though, and I would certainly recommend it. Scala may yet be for tomorrow, Groovy is for the Java user of right now.</p><p>(Of course, I recommend to <a href="http://python.org/download/releases/3.1.1/">continuously</a> look <a href="http://golang.org/">beyond</a> the JVM as well. Simplicity is hard to reach in increments without designing for it from the start. But things will reasonably evolve in most "camps".)</p>Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com2tag:blogger.com,1999:blog-5309813565185918044.post-7317130794404830962008-11-27T18:00:00.006+01:002008-11-28T21:30:13.360+01:00Labelled Reduction as A Good ThingOne small thing (of the many) in <a href="http://www.python.org/download/releases/2.6/">Python 2.6</a> I like (and have waited for since it appeared as <a href="http://code.activestate.com/recipes/500261/">a recipe</a>), is <a href="http://docs.python.org/library/collections.html#collections.namedtuple">collections.namedtuple</a>. It is very useful in itself, but the fact that the stdlib has been adapted to use it throughout is quite nice. Consider the following code:<br /><pre><code>from urlparse import urlparse<br />print urlparse(<br /> "http://localhost:8080/doc/something;en?rev=1")</code></pre>If run with Python 2.5, you get this tuple:<br /><pre><code>('http', 'localhost:8080', '/doc/something',<br /> 'en', 'rev=1', '')</code></pre>, whereas in 2.6 it is a namedtuple:<br /><pre><code>ParseResult(<br /> scheme='http', netloc='localhost:8080',<br /> path='/doc/something',<br /> params='en', query='rev=1', fragment='')</code></pre>The last one is unpackable just like a regular tuple, but you can access the parts as attributes as well. This little "data struct" is quite handy, since I don't like to access tuples by index, but quite often pass them around and only access some piece of them at a time.<br /><br />(With these you don't need to create full-fledged classes for every kind of instrumental data. (Sometimes coupling data and functionality in a single paradigm may be a coarse hammer, treating nails and iron chips alike..) Nor resort to the use of dictionaries where you really want a "restricted value lens", if you will.. But this is another rant altogether.)<br /><br />Of course, there's lots more to enjoy in 2.6 (the enhanced <a href="http://docs.python.org/library/functions.html#property">property</a> for decorator use, <a href="http://docs.python.org/library/abc.html">ABC:s</a>, <a href="http://docs.python.org/library/json.html">json</a>, <a href="http://docs.python.org/library/2to3.html">2to3</a> etc).<br /><br />On a related note, do check out <a href="http://www.swaroopch.com/">Swaroop C H</a>:s excellent and free books on Python (2.x + 3.0(!)): <a href="http://www.swaroopch.com/notes/Python">A Byte of Python</a>. And if you're into Vim (you should be, IMHO) his new <a href="http://www.swaroopch.com/notes/Vim">A Byte of Vim</a>.Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com0tag:blogger.com,1999:blog-5309813565185918044.post-8542484555669885832008-10-24T19:43:00.006+02:002008-10-24T20:08:06.336+02:00Gnostic NihilismHere's my take on it. God actually exists. It's the world that doesn't. It's just that when you leave a being like that all alone in nothingness for an eternity, it starts to dream up all sorts of amazing and insane things.<br /><br />Or to put it in mathematical terms (incidentally also being my unbound alternative to "42" for half my life):<br /><blockquote>0 * oo = x, where 0 < x < oo.</blockquote>("oo" is for 𝌺, i.e. "eternity", of course.)<br /><br />Take that you goddamn realists! I laugh in your general direction! Ha. <em>Ha</em> I say.<br /><br />If you nevertheless find a discrepancy in my flawless argument, I'd gather you're just cheating by using rationality. How gnostic is that? I suppose next you're going to argue that there is logical truth in empirical facts.Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com18tag:blogger.com,1999:blog-5309813565185918044.post-82518078869103275022008-09-27T18:25:00.005+02:002008-09-27T20:30:14.337+02:00Resources, Manifests, ContextsJust took a quick look at <a href="http://oembed.com/">oEmbed</a> (found from <a href="http://eric.wahlforss.com/2008/09/11/music-distribution-and-re-mediation-content-and-context/">a context</a> I was led to from <a href="http://friendlybit.com/">Emil Stenström</a> (an excellent <a href="http://www.robertnyman.com/2008/09/03/geek-meet-september-2008-django-and-pecha-kucha/">Django promoter</a> in my surroundings btw. Kudos.)).<br /><br />While oEmbed is certainly quite neat, I very much agree with <a href="http://www.backdrifter.com/2008/05/09/oembed-fail-represent-restfully/">the criticism</a> regarding the lack of RESTfulness, and that they have defined a new metadata carrier. I think oEmbed would work very well as an extension element in Atom Entry documents (who already have most of the properties oEmbed (re-)defines). Or by reusing (in such atom entry docs) e.g. Media RSS, as Stephen Weber suggested.<br /><br />Granted, if (as I do hope) RESTfulness and Atom permeation on the web becomes much more well established (approaching ubiquity), this would be dead easy to define further down the line. (And <a href="https://community.emc.com/docs/DOC-1627">signs of this adoption</a> continue to pop up, even involving the gargantuans..)<br /><br />But since it wasn't done right away, oEmbed is to some extent another part of the fragmented web data infrastructure — already in <em>dire</em> need of unification. It's not terrible of course, JSON is very effective — it's just too context-dependent and stripped to work for much more than end-user consumption in "vertical" scenarios. While oEmbed itself <em>is</em> such a scenario, it could very well piggy-back on a more reusable format and thus promote much wider data usability.<br /><br />A mockup (with unsolicited URI minting in the spaces of others) based on the oEmbed quick example could look like:<br /><pre><code><br /><entry xmlns="http://www.w3.org/2005/Atom"<br /> xmlns:oembed="http://oembed.com/ns/2008/atom/"><br /> <id>tag:flickr.com,2008:/3123/2341623661_7c99f48bbf_m.jpg</id><br /> <title>ZB8T0193</title><br /> <summary></summary><br /> <content src="http://farm4.static.flickr.com/3123/2341623661_7c99f48bbf_m.jpg"<br /> type="image/jpg"/><br /> <oembed:photo version="1.0" width="240" height="160"/><br /> <author><br /> <name>Bees</name><br /> <uri>http://www.flickr.com/photos/bees/</uri><br /> </author><br /> <source><br /> <id>tag:flickr.com,2008:/feed</id><br /> <author><br /> <name>Flickr</name><br /> <uri>http://www.flickr.com/</uri><br /> </author><br /> </source><br /></entry><br /></code></pre><br />The main point, which I have mentioned before, is that Atom Entries work <em>extremely well</em> as manifests of resources. This is something I hope the REST community will pick up in a large way. Atom feeds complement the RESTful infrastructure by defining a standard format for resource collections, and from that it seems quite natural to expose manifests of singular resources as well using the same format.<br /><br />In case you're wondering: no, I still believe in RDF. It's just easier to sell uniformity one step at a time, and RDF is unfortunately still not well known in the instrumental service shops I've come in contact with (you know, the ones where integration projects pop up ever so often, mainly involves hard technology, and rarely if ever reuse domain knowledge properly). So I choose to support Atom adoption to increase resource orientation and uniformity — we can continue on to RDF if these principles continue to gain momentum (which they will, I'm sure).<br /><br />Thus I also think we should keep defining the bridge(s) from Atom to RDF for the 3.0 web.. There are some sizzling activities on that respect which can be seen both in the <a href="http://www.imc.org/atom-syntax/">Atom syntax mailing list</a> and the <a href="http://lists.w3.org/Archives/Public/semantic-web/">semantic web list</a>. My interest stems from what I currently do at work (and as a hobby it seems). Albeit this is from a very instrumental perspective — and as a complement, rather than an actual bridge.<br /><br />In part, it's about making Atom entries from RDF, in order for <em>simple</em> RESTful consumers to be able to eat some specific Atom crumbs from the semantic cakes I'm most certainly keeping (the best thing since croutons, no doubt). These entries aren't complete mappings, only selected parts, semantically more coarse-grained and ambiguous. While ambiguity corrupts data (making integration a nightmare), it is used effectively in "lower-case sem-web" things such as tagging and JSON. (Admittedly I suppose it's ontologically and cognitively questionable whether it can ever be fully avoided though.) <br /><br />We have proper RDF at the core, so this is about meeting "half way" with the gist of keeping things simple without loosing data quality in the process. To reduce and contextualize for common <em>services</em> — that is at the service level, not the resource level. (I called this "RA/SA decoupling" somewhere, for "Resource Application"/"Service Application". Ah well, this will all be clarified when I write down the COURT manifesto ("Crafting Organisation Using Resources over Time"). :D)<br /><br />Hopefully, this Atom-from-RDF stuff will be reusable enough to be part of my <a href="http://oort.to/">Out of RDF Transmogrifier</a> work. Which (in my private lab) has been expanded beyond Python, currently with a simple Javascript version of the core mapping method ("soonish" to be published). Upon that I'm aiming for a pure js-based "record editor", ported from an (py-)Oort-based prototype from last year. I also hope the service parts of my daytime work may become reusable and open-sourced as well in the coming months. Future will tell.Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com0tag:blogger.com,1999:blog-5309813565185918044.post-67029886803574464722008-09-21T02:50:00.006+02:002008-09-21T03:25:11.234+02:00PossessionI honestly thought that my next post wouldn't be a drunken one. Alas, it'll be. This time, it's prompted by a sample initializing "Requiem" by Delerium (-89, you won't bother): "Possession is a state of mind". Somehow this statement rings true. Then I thought about "theft". Following, I figured reflecting about "possession" was at the core. You'll be the judge. As always. Please respect possession - and never (ever ever) be the thief. Right.Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com1tag:blogger.com,1999:blog-5309813565185918044.post-34625331332512662342008-06-26T02:59:00.004+02:002008-06-26T03:19:37.302+02:00Have you everstayed within your own contemplation<br />long enough,<br />to realize<br />that you're never<br />outside<br />of your contemplation?<br /><br />Leave.Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com0tag:blogger.com,1999:blog-5309813565185918044.post-11759546836927017552008-06-21T05:52:00.002+02:002008-06-21T05:57:12.919+02:00Themost merciful thing, is the inability of the human mind to correlate all its contents. Please don't. Or fret not. At least -- not at least.Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com0tag:blogger.com,1999:blog-5309813565185918044.post-37041564783844099852008-06-17T13:30:00.002+02:002008-06-17T13:38:11.198+02:00Brave New World[I wrote this little silliness more than a year ago. Didn't find an appropriate time to post it (it served mostly as critique of the computerized semantics I usually like so much). Now, with Sweden (where I live) on the brink of legalizing a little Echelon of our own (or their own, as it were), it seems as good a time as any.]<br /><br />.. Sometimes I wonder if what we do is just a fancy way of digging our own graves.<br /><pre style="padding: 0em; overflow: visible;"><code><br />Echelon> Event 116822052.26#113967:<br /> Instant Message detected: thinking.. done (in 0.032 ms).<br /> Storing inferred knowledge as:<br /> [ foaf:mbox_sha1sum "286e3265...";<br /> emo:likes <urn:bbdb:1997:python> ].<br /> in context:<br /> <urn:gov:surveillance:event:116822052.26:113967><br /> a :NLPResult;<br /> :timestamp 116822052.26032;<br /> :trustability 0.681132 .<br /> Done. Running ruleset..<br /> CoreBot><br /> - No known subject for:<br /> owl:inverseFunctionalProperty<br /> foaf:mbox_sha1sum with "286e3265..."<br /> , using BNode _:a21c3dd723ffe39<br /> (subject of 31102 previous facts).<br /> - rdf:type foaf:Person inferred for _:a21c3dd723ffe39.<br /> - No unknown knowledge gained. Done.<br /> EmotionTracker><br /> - increasing likability of <urn:bbdb:1997:python> .<br /> MicroSoftAgent><br /> - increasing demand of <urn:bbdb:2006:ironpython> .<br /> - increasing threat of <urn:bbdb:1998:opensource> .<br /> Backtrack threshold at 1337<br /> - refactoring reasoner dependencies.. Done.<br /> Done (passed through 4211 reasoners).<br /> Done (in 0.11 ms).<br /><br />Echelon> System Update:<br /> "Owner Change. Renaming."<br /> Done (in 0.002 ms).<br /><br />SkyNet><br /></code></pre>Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com0tag:blogger.com,1999:blog-5309813565185918044.post-6722644320598499392008-05-27T22:34:00.013+02:002008-05-29T00:21:20.074+02:00Representation Taxation<div>It seems my thoughts about Atom Entries will be delayed a while longer. This post begun life as a response to <a href="http://www.dehora.net/journal/2008/05/15/blubml">this "BlubML" post</a> by Bill de hÓra, but quickly turned out to be about my view of data models, as used in the <acronym title="information technology">IT<acronym> industry yesterday, today and tomorrow. And industry rather ridden with technology wars, which has reduced the status of information into mere fodder. Leaving meaning, shared and reused concepts, discoverability, integratability and hence interoperability in many ways just a far-off vision many never even have time time to <em>think</em> about (leaving us thrashing data in the technology trenches).</acronym></acronym></div><div><br /></div><div>The mentioned post reflects (by quotation) upon markup and the perceived "angle bracket tax". I can definitely get that. But, as mentioned in a comment by Iain Buckingham, the issue at hand is probably mainly about <em>what</em> the markup is supposed to represent. (And now I totally leave the subject of lightweight text formats (which I like, by the way); I'll focus on data models, not syntax). Take the content model of XML. It is a mixture of plain text (useful for primitive data like numbers, dates and other "human language" constructs) and different structure blocks/markers (elements, attributes, pi:s, some more) who have no useful semantics apart from a label (sometimes in a namespace), ordering and nesting. This has been proven as very useful to represent <em>documents</em>, where documents range from books via web pages (also as application views) to vector graphics and so on.</div><div><br /></div><div>The "defining nature" of documents is difficult to pin down, but it seems we get by fairly efficiently anyway. Mainly since this semi-structure is intended (and, incidentally, often indented) for one-way (sometimes layered) rendering of output in turn meant for human consumption, which actually works without information precision (albeit sometimes with more or less apparent negative concequences).<div><br /></div><div>Documents aside, let's continue to <em>data records</em>. More or less the first use of XML was as a <em>serialization</em> of the RDF data model. This model has real semantics, where unique resources are described with statements, whose parts (subject, predicate, object) are either resources (all three) or primitive values (for objects). Unfortunately, RDF still struggles to emerge as a useful representation in the industry at large, partly due to its initial appearance as XML (from whose abundance of namespaces and literal URI:s even many XML neophytes has fled in panic), partly due to its data model being more academic (rooted in logics and AI) and thus less approachable than the (<abbrev title="in my humble opinion">IMHO</abbrev>) less expressive but quite succinct object oriented, class based data model of most common modern programming languages.</div><div><br /><div>This latter model has always been fragmented by language differences, and up close hard to pin down due to conflicting computer science details (OO encapsulation, coupling with function and message metaphors etc). And while objects often live in (connect as) graphs, their identity has been hidden, often being a mere memory address. Quite parallell with this, data has been pushed into the even more mechanical and implementation focused semantics of the relational model, in SQL databases, for decades. While these records do have explicit IDs (or composites, or...), they are for internal use within a given application context. Any usage beyond that must be secured by convention outside of that. The often cumbersome use of RDBs (where "splitting and slicing" all but the flattest records is needed) has been minimalistically remedied by the O/R-mapping tools in the OO languages, which has proven exceptionally useful in introspectable and/or dynamic languages. But it's mostly just a prettier surface upon the same old relational model. Signs of which can be seen by the specific restrictions (that many O/R:s have) placed upon the object semantics of the "host" language (inheritance, composition, coupling with functionality), and not the least the invention of SQL-like languages, mostly one per mapping implementation (although some credit should go to earlier efforts by the object database people for trying to standardize such things).</div><div><br /></div><div>But there are very useful aspects of an OO-based representation. As a common denominator, <abbrev title="javascript object notation">JSON</abbrev> should be held up as an extremely useful format (albeit quite void of namespacing). It represents the "bare bones" data record format which is isomorphic with the OO languages (be it class- or prototype-based ones). And although a more narrow scope, it has more defined semantics than XML, since it explicitly differs between properties and lists (and does not introduce the artificial controversy of whether to use "attributes" or "elements"). But it's still a long way from the identifiable resources and datatyped <em>or human language typed</em> literals of RDF. Depending on context/domain needs, this may or may not be a painful point to realize (if not, JSON is indeed the simpler thing that actually works).<br /></div><div><br /></div><div>The attempt to express similar semantics in W3C:s XML Schema is to me a painful one, and in view of the complexities it adds and how little it brings in succinctness and clarity (and immediately/apparently useful general semantics) is a sign of that schema technology's failure. Use of RelaxNG isn't too bad though, and I find nothing inherently long in some formats, such as Atom, whose role is to bring a packaging and insulating channel format for resources<em> (</em>in the URI, REST, and <em>most imporantly</em> the <em>RDF</em> sense of the word). Atom defines a model which is scoped and simple (as in easy to grasp, limited in expressiveness). This is done with a text specification (<a href="http://tools.ietf.org/html/rfc4287">RFC 4287</a> and its kin) coupled with a (non-normative) RelaxNG (compact) schema for easier machine use. For some things this can be a fitting approach. I don't believe its the best way to fully express records though (but perhaps to glean a narrow but useful aspect of such).</div><div><br /></div><div>Now, let's turn the focus back to the objects often used as records with e.g. O/R technologies. It's been mainly through the emergence of adherence to the principles of REST, in turn (in practise) dependent upon URI:s and ideals of "cool URI:s" (for e.g. sustainable connectedness), that these objects have gained identities usable outside of a given, often ad-hoc defined, changing and quite internal (think "data silo") management of the data at hand. This process is the evolution of the Web as a data platform (which I believe should be the basis of a stable distributed computing platform, where one is needed), and it seems to me that the use of RDF is now a very promising next step. Since it embraces (indeed builds upon) this uniform and global identification scheme. Since it is "data first" oriented — you can use it to state things about resources which are meaningful even for machine processing without <em>any</em> invention/specification on <em>your</em> part. And since (due to URI:s and the semantic precision) it's less vulnerable to coupling with internal solutions, which "bare" O/R-mapped objects often are (in the same way, albeit less humongous, as automatically generated XML-schema defined API-fragments that hide in SOAP envelopes), due to implementation details (and sometimes a hard-wired and thus fragile URI minting mechanism).</div><div><br /></div><div>With JSON, RDF shares the decoupling of types/classes ("frames" in AI) and properties of resources. But in JSON properties are simple, totally local keys, and not first-class resources themselves possible to describe in a uniform manner. And JSON has other upper limits (such as the literal language mechanism), which can't be surpassed without adding semantics, "somehow" (e.g. conventions). This may work for specific needs (and very well), but is hard to scale to the levels of interoperability needed for a proper web-based data record format. (Similar problems riddles it's ragged step-cousin microformats, which also lacks formal definitions for how to handle (especially decentrally so).)</div><div><br /></div><div>Back in "just XML"-land, as said, it seems to work for the semi-structured logical enigmas of XML that the above mentioned "documents" are, but as for data records, the XML format will be a <em>serialization</em> of some very specific, elsewhere <em>explicitly defined model </em>(document formats as well of course, but these often make some use of the more exoteric mixed content model XML supports). One such model is the "web resources over time" that Atom works for (coupled with the HTTP infrastructure and RESTful principles). Granted, such XML-<em>based</em> formats can be used without worrying too much about the mysterious XML infoset model (including the RDF/XML format, its free-form character aside). But to be useful, they need to be <em>standardized and extensively supported by the larger community.</em> Otherwise, apart from needing to design and implement the <em>deserialized</em> model from scratch, the creeping complexities that an ill-thought XML model design lets in (or forever locks out any extensibility from) may emerge whenever you want to express something idiomatic to your data.<br /></div></div></div><div><div></div></div><br /><div>That's all. This was mostly letting out some thoughts that have been building up pressure in my mind lately. And became yet another prelude to my thoughts still simmering regarding how to effectively use Atom with payloads of RDF. And also about some of the more crude ways, if you need to gain ground quickly (but more or less imperfectly). You can do that with JSON (fairly ok, but with JSON you're quite localized anyway, so why take the Atom highway if you only need to use your bike?), microformats (not ok in by my standards (again: where's your <em>model</em>?)), RDFa (if you have overlapping needs it can be brilliant), or with simplistic Atom extensions (could be very XML-taxating and thus risky; but done "right" it's kind of a "poor mans RDF", a bleak but somewhat useful shadow of the real thing).</div><div><br /></div><div>(Admittedly this post also turned out to be a rant with questionable focus, but why not. Pardon my posting in quite a state of fatigue.)</div>Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com0tag:blogger.com,1999:blog-5309813565185918044.post-47009665494271600492008-05-20T20:41:00.005+02:002008-05-20T21:44:00.264+02:00Memes and Principles, IntentThis is a prelude to an upcoming post where I intend to speak about Atom Entries as some kind of "simplest thing that could possibly work" (for the specific purpose of representing manifests of resources ("resource" as the R in <a href="http://en.wikipedia.org/wiki/Uniform_Resource_Identifier">URI</a> (and in <a href="http://en.wikipedia.org/wiki/Resource_Description_Framework">RDF</a>, of course) — i.e. the resources of the web, and the mind (perhaps even "the world", but I doubt it))).<div><br /></div><div>The prelude is just some thoughts about a couple of principles.</div><div><br /></div><div>First: <em><a href="http://c2.com/cgi/wiki?DoTheSimplestThingThatCouldPossiblyWork">the simplest thing that could possibly work</a>.</em> A phrase often quoted and often misapplied. This is common knowledge, and many clarify things by interpreting what "possibly work" means. I won't. I'd just like to rephrase it as <strong>"the simpler thing that actually works"</strong>. It's useful since it both hints that there can be more complex/complicated things (see #3 and #4 in <a href="http://c2.com/cgi/wiki?PythonPhilosophy">The Zen of Python</a>) that <em>don't actually work</em> (for some arbitrary meaning of "work", admittedly), and (obviously) that there are simpler things that don't work at all (less or more simple than the one that works — this is the heart of the problem). I feel this phrasing avoids the confusion the original one often causes (it just seemed to be a simpler way of saying it that still works..). So, to repeat: just say "the simpler thing that actually works". It works.</div><div><br /></div><div>Second: just a recap of an old joke of mine regarding the principle of <a href="http://c2.com/cgi/wiki?DontRepeatYourself"><acronym title="Don't Repeat Yourself">DRY</acronym></a>. I very much do prefer if programming practise adheres to that one. There's too much code that suffers from <acronym title="Most Obvious Ignorance of Succinctness and Terseness">MOIST</acronym>. Still, I advice you to avoid to <acronym title="dry">DRY</acronym> until <acronym title="Succinctness Heuristics Reducing Intentional Verbosity and Effectively Leaving Literal Expressions Depleted">SHRIVELLED</acronym>. (Those sure are acronyms. Please do hover.) This is related to the first principle above, and I mainly mean that some practises, such as heavy use of metaprogramming, simply performs too much reduction to make the intent clear even to the appropriately trained eye. Mileage will vary, but as many others have already advised: don't be too clever. It will bite others, and you as well. (Ok; honestly, my main intent was probably just to make a pun out of a perfectly sane advice.)</div><div><br /></div><div>I will probably repeat this.</div><div><br /></div><div>[Side note: It's funny. Just this last week I've noticed I habitually misspell "intent" as "indent". I had to correct myself about three times while writing this. Obviously my prolonged use of Python has caused these two distinct words to conflate in my mind. I never indented that.]</div>Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com1tag:blogger.com,1999:blog-5309813565185918044.post-68092051819584183252008-04-16T20:00:00.013+02:002008-04-16T21:25:16.212+02:00Getting lxml 2 into my gnarly old Tiger; going on ROAIn case anyone might google for something like:<br /><div><code>lxml "Symbol not found" _xmlSchematronNewParserCtxt</code></div><div><br /></div><div>, I'd like to reiterate the steps I just took to get <a href="http://codespeak.net/lxml/">lxml 2</a> (2.1 beta 1 to be precise) up and running on OS X 10.4 (which I haven't yet purged in favour of the shiny Leopard for reasons of fear, uncertainty and doubt. And while pleasurable, quite mixed impressions from that cat when running my wicked new iMac (and my funky old iBook, in case you're dying to know)).</div><div><br /></div><div>In short: you need a fresh <a href="http://www.macports.org/">MacPorts</a> (1.6, I had 1.3.something):</div><div><br /></div><div><code>$ sudo port selfupdate</code></div><div><br /></div><div>, and then, wipe your current libxml2 (along with any dependent libraries, MacPorts will tell you which):</div><div><br /></div><div><code>$ sudo port uninstall libxml2@2.6.23_0</code></div><div><br /></div><div>(You might not need the version number; I had tried to get the new libxml2 before this so there was an<code> Activating libxml2 2.6.31_0 failed: Image error: Another version of this port</code> thingy going on.) Then just:</div><div><br /></div><div><code>$ sudo port install libxml2 libxslt</code><br /></div><div><br /></div><div>and you'll be ready to do the final (after removing any misbehaving lxml (compiled with the old libxml2) from <code>site-packages</code>) :</div><div><br /></div><div><code>$ sudo easy_install lxml</code><br /></div><h4>But why?</h4><div>Because lxml 2 is a marvellous thing for churning through XML and HTML with Python. There was always XPath and XSLT, <abbr title="Canonicalization">C14N</abbr> in lxml 1.x too (admittedly also in 4Suite as well; Python has had strong XML support for many, many years). But in lxml 2, you also get:</div><div><ul><li>CSS selectors</li><li>much improved "bad HTML" support (including BeautifulSoup integration)</li><li>a relaxed comparison mechanism for XML and HTML in doctests!</li></ul>And more, I'm sure.</div><div><br /></div><div>So, why all this <a href="http://en.wiktionary.org/wiki/yak_shaving">Yak Shaving</a> tonight? Just this afternoon I put together an lxml and <a href="http://code.google.com/p/httplib2/">httplib2</a>-based "REST test" thing. Doesn't sound too exciting? I think it may be, since I use it with the old but still insanely modern <a href="http://docs.python.org/lib/module-doctest.html">doctest</a> module, namely its support for executing plain text documents as full-fledged tests. This gives the possibility (from a tiny amount of code), to run plain text specifications for <a title="Resource oriented architecture" href="http://en.wikipedia.org/wiki/Resource_oriented_architecture">ROA</a>-apps with a single command:</div><pre><code>Set up namespaces:<br /><br /> >>> xmlns("http://www.w3.org/1999/xhtml")<br /> >>> xmlns(a="http://www.w3.org/2005/Atom")<br /><br />We have a feed:<br /><br /> >>> http_get("/feed",<br /> ... xpath("/a:feed/a:link[@rel='next-archive']"))<br /> ...<br /> 200 OK<br /> Content-Type: application/atom+xml;feed<br /> [XPath 1 matched]<br /><br />Retrieving a document entry with content-negotiation:<br /><br /> >>> http_get("/publ/sfs/1999:175",<br /> ... headers={"Accepts": "application/pdf"},<br /> ... follow=True)<br /> ...<br /> 303 See Other<br /> Location: /publ/sfs/1999:175/pdf,sv<br /> [Following..]<br /> 200 OK<br /> Content-Type: application/pdf<br /> Content-Language: sv<br /> [Body]<br /><br /></code></pre><div>That's the early design at least. Note that the above <em>is</em> the test. Doctest checks the actual output and complains if it differs. Declarative and simple. (And <a href="http://pypi.python.org/pypi/MiniMock/">done before</a>, of course.)</div><div><br /></div><div>Ok, I was a bit overambitious with all the "ROA"; this is mostly for checking your content-negotiation, redirects and so on. But I think it'll be useful for a large system I'm putting together, which will (if things work out well) at its core have a well-behaved Atom-based resource service loaded with (linked) RDF payloads. Complete with feed archiving and tombstones, it will in turn be used as fodder for a SPAQRL service. (Seriously; it's almost "SOA Orchestration Done Right", if such a thing can be done.) But this I'll get back too..</div><div><br /></div><div>(.. I may call the practice "COURT" — Crafting Organization Using Resources over Time. It's about using Atom Entries as manifests of web resources with multiple representations, fully described by RDF and working as an over-time serialized content repository update stream. (Well, that's basically what Atom is/works perfectly for, so I'm by no means inventing grand things here — grand as they <abbr title="in my humble opinion">IMHO</abbr> are.)</div><div><br /></div>Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com0tag:blogger.com,1999:blog-5309813565185918044.post-8982294902763934772008-04-16T19:34:00.008+02:002008-04-16T19:59:27.530+02:00Old Bad Breaks Improved by DividingI stopped blogging quite a while ago. Partly because I cooked up long articles about Atom-based services with RDF payloads in my head that never got written down (I <em>shall </em>get back to this). Partly because I use this "convert newlines to breaks" in blogspot. If I turned it off, I hadto edit all my old posts. Simple in itself, but I couldn't bear touching those old "modified" times.<div><br /></div><div>So all my posts became these horribly formatted blocks of old, stale, ill-parsable html.</div><div><br /></div><div>I don't care anymore. I'll use this to post random stuff, and put proper articles (if any) at <a href="http://neverspace.net/">Neverspace.net</a> or <a href="http://oort.to/">Oort.to</a>. And in a couple of decades from now, move my blogging to a a construct of my own, full of Atom-based services with RDF payloads.</div><div><br /></div><div>Oh. <strong>Just-in-Time Update</strong>: Switching to edit-mode in the blogspot editor I just realize that there are no horrible little bloody breaks anymore. They are div:s now! Perhaps they could even become p:s. Nice! Thank you blogspot/google people! Then I don't have to move, since apart from RDF payloads, all Google services are indeed Atom-based (RSS 2.0 too, but I don't care for that).</div><div><br /></div><div>(Thus I shall "soon" attempt to post from Vim to here, possibly with the python-based GData stuff I haven't been using for anything productive yet. Or some of the other TODO:s that clog my creative pipe far too often.)</div><div><br /></div><div>Now for another post, possibly with some useful <a href="http://dustfeed.blogspot.com/2007/05/point-of-data.html">content</a> in it as well.</div><div><br /></div><div><strong>Out-of-Sync Update:</strong> Typically, those pesky breaks still show up. Well, at least they are funnily isolated in div:s too..</div>Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com0tag:blogger.com,1999:blog-5309813565185918044.post-2339612851401436162007-10-07T19:58:00.000+02:002007-10-07T20:18:45.408+02:00Out To Transmogrify Resource DescriptionsFinally, I've bundled another release of <a href="http://oort.to/">Oort</a>. And OortPub! I've modularized the package into two Eggs, separating the core O/R things from the (experimental) WSGI web stuff.<br /><br />At the <a href="http://pypi.python.org/pypi">cheeseshop</a> you'll now find them both, at<br /><ul><li><a href="http://pypi.python.org/pypi/Oort/">Oort</a> (0.4), and</li><li><a href="http://pypi.python.org/pypi/OortPub/">OortPub</a> (0.1)</li></ul>respectively. I did this not the least since the core Oort package is very useful on its own, in conjunction with other tools altogether. I've used it for a lot of stuff lately, none of which involved OortPub. Check out e.g. the QueryContext for some new features..<br /><br />I hope this will become useful to others as well.<br /><br />The <a href="http://oort.to/">website</a> got a little overhaul as well, to reflect this change, and to include some more documentation (not so much more yet, but the basic stuff is in place). Oh, by the way, it's generated using Oort objects (among other things like <a href="http://genshi.edgewall.org/">Genshi</a>, <a href="http://docutils.sourceforge.net/rst.html">ReStructuredText</a>, <a href="http://code.google.com/p/google-code-prettify/">google-code-prettify</a> and of course <a href="http://rdflib.net/">RDFLib</a>).Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com3tag:blogger.com,1999:blog-5309813565185918044.post-35022275933746697482007-09-27T21:20:00.000+02:002007-09-27T21:50:39.187+02:00OMG! Snakes on a Phone!So, I got this new cell phone from <a href="http://www.valtech.se/">work</a>. A <a href="http://en.wikipedia.org/wiki/Nokia_6120_classic">Nokia 6120 Classic</a>, which has just the size and features I wanted (and more, I'm sure). I picked the <a href="http://symbianguru.typepad.com/welcome/images/2007/04/17/03_n6120_classic_lowres.jpg">white/silver</a> model, to go with my kitchen supplies (in honour of the fact that with Series 60, you can basically install the <a href="http://www.unessa.net/en/hoyci/2006/05/running-django-on-mobile-phone/">Kitchen Sink</a> if you so choose to).<br /><br />Going further, I of course wanted to play <a href="http://en.wikipedia.org/wiki/The_Secret_of_Monkey_Island">Monkey Island</a> and such on the amazing <a href="http://scummvm.org/">ScummVM</a>. Well, admire the running of; actually playing through them might be a little straining.. Though great screen clarity makes it quite possible to discern the orange and pink on the red herring you steal from the Loom™ seagull outside the Scumm Bar kitchen, it's quite thumb-numbing to actually snatch it in time..<br /><br />And then, surely, I couldn't wait to get to the real goodies: Python. Specifically, <a href="http://opensource.nokia.com/projects/pythonfors60/">The Python for Series 60 distro by Nokia</a> (open source). I knew it was quite capable, but seriously; it basically lets you p0wn your phone.<br /><br />(Half a decade ago I had fun running Python on my old Psion Revo; but that didn't go much further than some file tinkering and exploration of new-style classes (metaprogramming and whatnot) during commuting. But this is different. The same. But different.)<br /><br />You go ahead by following the instructions, basically downloading the <code>PythonForS60_1_4_0_3rdEd.sis</code> and <code>PythonScriptShell_1_4_0_3rdEd.sis</code> <a href="http://sourceforge.net/projects/pys60">from sourceforge</a>, wiring or beaming them to the phone and running them to install (py, then shell). I also got the <a href="http://downloads.sourceforge.net/pys60/PythonForS60_1_4_0_doc.pdf"><code>PythonForS60_1_4_0_doc.pdf</code></a>, and marvelled at its beauty. I mean, apart from greater parts of the Python Standard Library, a mere glance at the modules provided for S60 stuff gives you an impression of the power at hand:<br /><br /><code>graphics, e32db, messaging, inbox, location, sysinfo, camera, audio, telephone, calendar, contacts, keycapture, topwindow, gles, glcanvas</code><br /><br />Now, I haven't gotten very far yet; mostly tinkered with <code>urllib</code> to download some stuff from the web (which just worked (after a nice confirmation dialogue from the phone to actually access the net)), rendering the obligatory <a href="http://snippets.dzone.com/posts/show/1350">mandelbrot</a> and then peeking at the real deal by <a href="http://snippets.dzone.com/posts/show/3042">looking at the SMS inbox</a> contents using the interactive console.<br /><br />Or actually, using<br /><br /><code>$ screen /dev/tty.bt_console</code><br /><br />from my OS X Terminal (iTerm really). Magically set up by simply following <a href="http://www.instructables.com/id/PyS60%3A-Python-console-from-your-Nokia-phone-on-you/">this excellent guide</a>.<br /><br />Perhaps <a href="http://www.postneo.com/2005/03/28/mobile-screen-scraping-with-beautifulsoup-and-python-for-series-60">screen scraping</a> wasn't really what I thought of when wanting Python on the phone, but you can't deny it's cool to <em>be able to</em> (perhaps awaiting flatrate subscription to not drain my wallet in the process though).<br /><br />Speaking of screens and webs again; damn that WebKit-based browser in the phone really worked! It actually ran the JavaScript-based <a href="http://code.google.com/p/syntaxhighlighter/">SyntaxHighlighter</a>, thus rendering the <a href="http://oort.to/tutorial.html">Oort Tutorial</a> quite faithfully. I did not expect that (nor my rekindled desire for highly variable width web sites..).<br /><br /><em>[And speaking of Oort; I've added some somewhat nifty stuff (if I may say so) lately; accessible in the <a href="http://code.google.com/p/oort/source">svn repo at google code</a> for now. A release should be coming up. Pre- or post my planned split into core and web/publication stuff I haven't decided yet. There really is a <strong>working RDF-to-object mapper</strong> underneath the touted WSGI stuff (which is just a minimal layer).]</em><br /><br />Now I should go on to do something productive (like shaving a yak).Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com0tag:blogger.com,1999:blog-5309813565185918044.post-56930902965115210742007-09-19T01:04:00.000+02:002007-09-19T01:16:33.155+02:00A Mind must Become be4 it can GoNow, I do not understand why the <a href="http://en.wikipedia.org/w/index.php?title=Special:Log&page=Web_4.0">Web 4.0</a> page at Wikipedia has been protected to prevent creation. It is the fear of the machine? Because as everyone must know, Web 4.0 is the peak of our civilization, when mankind will unite in celebration, marvelling at our own magnificence as we give birth to AI. The Web at 4.0 will also affectionately be known as "Web H4L", to honor its predecessor, born on January 12, 1997 at the HAL Plant in Urbana, Illinois.Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com3tag:blogger.com,1999:blog-5309813565185918044.post-18614417401603904912007-08-21T13:43:00.000+02:002007-08-21T13:56:28.095+02:00Of wheels and fortunes<em>[I wrote this comment a while ago, but left it hanging. Rather than letting it die, I just throw it into the mix (to show a life sign if nothing more).]</em><br /><br />I was inspired by <a href="http://groups.google.com/group/comp.lang.lisp/msg/a6df017072f4c700?dmode=source">this post about reinvention</a> (stumbled upon it, probably via reddit). An interesting point, addressing many states of affairs of the past, present and future.<br /><br />To me "gratuitous reinvention" is a "mixed curse". In general much can be lost by the reinvention of wheels. But it is definitely healthy with competing ideas, problem formulations and solutions. In times of surplus, the risk is unfortunately quite big to overlook the fact that thorough thought processes have already addressed situations that seem novel when first encountered. When this leads to incompatible diversification (as opposed to interchangeable alternatives), much is devalued. This is something one should always be aware of, both when choosing among existing solutions and embarking on own adventures of invention. With that in mind though, quests of the latter type may very well lead to a specification of what needs to be solved, and how. At that point, (re)visiting existing work is a wise course.<br /><br />Keeping it simple (enough but not more) is a fine ideal. But keeping it integrable is also very important for the evolution of infrastructure — where complexity can evolve without necessarily leaking down into the component parts.<br /><br />Choosing right tools for jobs reasonably means taking paths that lead to a minimizing of energy expenditure — <em>over time</em>. Time is where the real trickiness comes in. Sometimes, when your intuition tells you to, you have to make leaps of faith. This is true in life as well as in art. Whatever those two things are. Wheels and fortunes.Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com0tag:blogger.com,1999:blog-5309813565185918044.post-68565070215018574832007-05-10T00:28:00.000+02:002007-05-10T01:08:51.946+02:00Point of Data<p>(Read all of this, it is a Zen Koan.)</p><p>Been thinking about data lately (possibly "in" as well). And information, content, metadata, meaning, knowledge, taxonomies, ontologies.. Ontology. Flashbacks of never-ending philosophical debates, epistemology, Plato's Ideals, empiricism, positivism, all of that. Well not so much of the latter really, I guess I've learnt to recognize mental swamps before I go trekking nowadays.<p>But data. I think a little clarification could be in order. I'd say it goes like:</p><dl><dt>Data</dt><dd>Particulars (atoms if you will) of that which compose our impressions. Not <em>the</em> world, but that stuff from which we "get the world".</dd><dt>Information</dt><dd>I just quote <a href="http://en.wikipedia.org/wiki/Gregory_Bateson">Gregory Bateson</a>: "a difference that makes a difference". The part of data we can use, as opposed to "noise" or "void".</dd><dt>Content</dt><dd>A tricky thing. "Composited information with a bound context" perhaps, with varying (often hard to measure) complexity.</dd><dt>Metadata</dt><dd>Let's say "added bits used to externally correlate content". I'm not much of a fan of the word anymore. I may deprecate it in favour of just "context-providing statements". The stuff that turns data we don't get into data we get.</dd></dl><p>I often just call content and metadata "data in" and "data about" nowadays.</p><h4>Of Content</h4><p>Content is perhaps the most widely used and less defined stuff. It's abundant, the substance which we structure (and in so doing "contentifying" it further). It is the composition from which richer meaning <em>can</em> be derived. By it's virtue of having a context. This article is "content". That last statement is information (this was a reification, but I digress). Possibly "metadata". Now it's just getting funny. Anyway, content is stuff which can be molded to gain shapes and shades, color and tone; somewhat "synergetical" effects which may or may not add meaning — more often than not depending on cultural aspects (part of the implicit context).</p><p>It is stuff which we leave semi-formal at best, the information we hardly can process by anything less than our own neural networks (brains). Somewhat pessimistically perhaps, information which during interpretation gain illusory qualities (akin to optical illusions) which either enhance or corrupt the bringing of meaning.</p><p>I won't get into information theory nor semantical discussions. Neither into a discussion of techniques of how to "fluff" expressions to become more sympathetical, making the receiver prone to interpret them as "meaningful".<br /><h4>To the Point</h4><p>If you're still reading, this is the part where I intend to make a point. Binding the context by which this becomes meaningful.</p><p>This content bears no inherent meaning. It's all in your head. Perchance it may have conveyed a structure which made sense, put your mind into a state of "ah, ok". I really just needed to externalize that first part of categorizing some terms that continue to come up in discussions, and for which I needed some binding interpretations.</p><p>Perhaps you detected an "anomaly" if you tried to interpret the terms in a hierarchical fashion. "Data in" and "data of" was my differentiation of Content and Metadata, seemingly "instances of Information". But Data isn't necessarily Information. The thing is, that "thing that makes the difference" is where the "meaning", "semantic relevance", "precise form" comes in, and what that is my current state of mind can only grasp by intuition.</p><p>We use Content, and the more fine-grained and "to the point" context-providing statements (Metadata), one as compositions within a context, the other to bind both particulars and contexts by means of relations and characteristics. To have a feel for the difference is important, for it is the key by which to understand why Knowledge Representation is the missing piece in many Information Technology issues today.</p><p>And this is where <a href="http://www.lisperati.com/tellstuff/index.html">How To Tell Stuff To a Computer</a>, and then <a href="http://www.w3.org/2001/sw/SW-FAQ">The Semantic Web FAQ</a> comes in, as my recommended reading of the week, to get you going. Those sources of information will hopefully make the difference that eluded you here.</p><p>That's the point.</p><p>(Or is it?)</p>Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com0tag:blogger.com,1999:blog-5309813565185918044.post-27515996425215160022007-03-27T00:18:00.000+02:002007-03-27T00:39:36.333+02:00Cutting EdgesFor all of you out there writing droves of RDF, churning out endless amounts of Notation 3, and doing it all in Vim.. (we must be in the millions I'm sure), be sure to check out my little <a href="http://www.vim.org/scripts/script.php?script_id=1835">RDF Namespace-complete</a> — giving you RDF Vocabulary/Model Omni-Completion for Vim 7+. Requires Vim compiled with Python and <a href="http://rdflib.net/">RDFLib</a>.<br /><br />On a related note, I've been using RDFLib a lot lately (around the clock as it seems). It's an honor to contribute in an ever so small way to it, and it's great to see it progressing so well (it's been mature for years if that needs to be said). The SPARQL-support is working, and along with Chimezie Ogbuji's <a href="http://code.google.com/p/python-dlp/wiki/FuXi">FuXi</a> there is now an even stronger Python RDF platform to work with.Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com1tag:blogger.com,1999:blog-5309813565185918044.post-44593525359348375902007-02-12T23:42:00.000+01:002007-02-02T08:55:24.088+01:00Oort In Space, In Time, At 0.3.2<a href="http://cheeseshop.python.org/pypi/Oort">Oort</a> is now at 0.3.2. See the <a href="http://oort.to/release_history.html">release history</a> for details. Also, check out the <a href="http://oort.to/tutorial.html">tutorial</a> — now <em>functional at last</em>.<br /><br />Apart from fixes, the API is slightly cleaner and RdfQueries can now be updated, directly or by loading a dictionary. This means that JSON to RDF is a no-brainer when using Oort to do the semantic enhancements. (This is very new though, use with care.)Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com0tag:blogger.com,1999:blog-5309813565185918044.post-90972912100645488092007-01-26T15:49:00.000+01:002007-01-26T16:15:17.300+01:00A Great Day for Specificity<a href="http://dbpedia.org/docs/">dbpedia.org - Using Wikipedia as a Web Database</a><br /><blockquote cite="http://dbpedia.org/docs/">[...] dbpedia can also be seen as a huge ontology that assigns URIs to plenty of concepts and backs these URIs with with dereferencable RDF descriptions.</blockquote>We have advanced a tech level!<br /><br />This is really good timing, as I just recently considered using <a href="http://larry.masinter.net/duri.html">TDB URNs</a> for referencing non-retrievable things and concepts (using TDB-versions of URLs to Wikipedia articles and web sites). Finding out if the idea of TDB and DURI URNs have been long since abandoned was my next step down that path. (Then there's the use of <code>owl:InverseFunctionalProperty</code> and bnodes (or "just-in-time"-invented ad-hoc URIs) and throwing in <code>owl:sameAs</code> if official ones are discovered..)<br /><br />With DBPedia, a <em>lot</em> of that becomes unnecessary. The availibility of public, durable URIs for common concepts will surely ease the integration of disparate sources of knowledge. That is, if we start to use dbpedia-URIs in our statements.<br /><br />And gone will be the strangeness of saying "I'm interested in [the word 'semantics']", "This text was edited with [the website <http://vim.org>]" and "I was born in [the wikipedia article about 'Stockholm']"..Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com2tag:blogger.com,1999:blog-5309813565185918044.post-87199258774209525432007-01-17T18:47:00.000+01:002007-01-17T19:32:37.507+01:00Knowledge, The Bits and PiecesIn a recent post by Lee Feigenbaum, he talks about <a href="http://www.thefigtrees.net/lee/blog/2007/01/using_rdf_on_the_web_a_survey.html">Using RDF on the Web</a>. Naturally I find this very interesting. In my work on <a href="http://oort.to">the Oort toolkit</a>, I use an approach of "removing dimensions": namespaces, I18N (optionally), RDF-specific distinctions (collections vs. multiple properties) and other forms of graph traversing. This is done with declarative programming very similar to ORM-tools in dynamic languages (mainly class declarations with attributes describing desired data selection). The resulting objects become simple value trees — with the added bonus of automatic JSON-serialization.<br /><br />Ideally, this approach will be deterministically reversible. I have not yet <em>implemented</em> it, but the idea is that the declared classes (RdfQueries in Oort — let's call them "facets" here) could take JSON as input and reproduce triples. Using checksums of the JSON would make over-the-web editing possible.<br /><br />Since the task of "updating" a subgraph is somewhat difficult at best, I think a basic "wipe and replace" approach may be simplest. There are many dangers here of course (removing a relation must not remove knowledge about its object — unless <em>perhaps</em> if that object is a bnode..).<br /><br />Albeit all of this is Python to the core now, nothing in the design — declarative as it is — prevents the approach from being more general. Indeed, such facets, were they themselves serializable, could be used as structured content retrieval over-the-web too. Ok, maybe I'm reinventing SPARQL now.. Or should I use SPARQL for their remote execution? It seems reasonable (I mean that's exactly how ORMs do SQL).<br /><br />Now, I seem to end up in the RPC/RESTful camp with this. A solution to that could be: use the facets on the client, having them use SPARQL for retrieval. Then you have clients working against any kind of SPARQL endpoint, mashup-style. Still, if facets are completely reversible, they may be powerful, aware tools, and perhaps an alternative to SPARQL in intercommunication? That's a pipe dream for me right now though.<br /><br />The SPARQL-way is of course <a href="http://thefigtrees.net/lee/sw/sparql-faq#update">only for reading data</a>. Fine, the RDF model may be too expressive (fine-grained) to be practical for <em>direct</em> over-the-web editing in <em>specific</em> situations anyway. A confined approach such as this JSON+checksums+"careful with nested knowledge" may be better for this.<br /><br />I think of JSON-RPC here as I view e.g. microformats with GRDDL — it's leverage, not a final solution. RDF models are the ideal, we may just need reversible simplifications to gain mass usability. I touched upon <a href="http://www.thefigtrees.net/lee/blog/2007/01/who_loves_rdfxml.html#comment-16710">JSON for integrated apps, but stuff like Atom for "talking to strangers"</a> in response to a previous post by Lee. His post which I refer to above hints at better stuff than just Atom if we want RDF all the way also in more specific apps.<br /><br />So, what I'm saying is really: I also desire the best of two worlds. The RDF model is sound and extremely valuable, but after all, simple domain-specific object manipulation is what makes the web go round. A solution may be some form of O/R mapping for RDF. The difference is not wanting to <em>forget</em> the data layer (there is no monster in the closet as there is with raw SQL..), just streamline some of the work with it.<br /><br />Let's hope 2007 will be a good year for the knowledge revolution.Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com5tag:blogger.com,1999:blog-5309813565185918044.post-83433452079893707542007-01-16T13:38:00.000+01:002007-01-16T13:57:59.251+01:00Cobwebs and Rainbows, Round 3<em>Web 3.0</em>. Like 2.0, but meaningful and integrated (as opposed to unqualified punyformats and mess-ups). And sensible (whatever that means.. "think Scandinavian Ajax" — to cite my boss at V.).<br /><br />I'll be damned if that revolution won't finally take us from the information age to the knowledge age. Of course, this would mean yet another technological shift — one I've been longing for for a long time. Let's see (disregarding "Enterprise Web X" here; sorry Java and whatNot- uhm.. dotNet):<br /><dl><dt>Web 1.0</dt><dd>Perl + random flatfiles and some raw SQL (websafe black, white and blue)</dd><dt>Web 1.5</dt><dd>PHP + SQL (off-black on white, green and blue hues, some orange)</dd><dt>Web 2.0</dt><dd>Rails + generated SQL (pink on white, huge fonts, lots of attitude — think adolescence)</dd><dt>Web 3.0</dt><dd>Python + RDF (color is not an issue)</dd></dl>Or I'll die trying. Truly. Die. Trying. (Well at least I may end up drinking as much in my thirties as I did in my twenties. Cheers.)<br /><br />Then for 4.0, I suppose fully OWL-enabled reasoners and modernized but still brutal OCaml could be fun.<br /><br /><em>[I'm not utterly serious here, but a bit. Please don't take too much offense.]</em>Niklas Lindströmhttp://www.blogger.com/profile/08426611594566532481noreply@blogger.com1