Monday, 23 February 2009

Hawai‘i, diacritics, and computer-meaningful statements in written language

A lovely piece on diacritics from pipwerks.

An ‘okina usually indicates a glottal stop, which is very important in the pronunciation of Hawaiian words.  The name Hawai‘i is a great example: the ‘okina indicates the name is pronounced hahwhy-ee instead of hahwhy. When you hear a native pronounce the name, there's usually a very short hard pause between the why and eesyllables.

Unfortunately, the two Hawaiian diacriticals are not used by European languages, which means they're difficult to accurately represent on a standard US qwerty keyboard. In most printed publications, the authors simply omit the diacriticals altogether — the very reason you usually see the name Hawaii, and not Hawai‘i.

It is easy for English speakers to forget that in many (if not most) languages, the funny little extra marks on the page aren't just there to look nice. They constitute meaningful units of language. There are multiple instances where their presence or abscence changes the meaning of the word (and one from Spanish is on the tip of my tongue, but I just can't catch it!) This has huge implications for computer use of language. There are few examples that apply to English, but there are still a few - 'can't' is a contraction of 'cannot', but 'cant' is a secret language (or an architechtural feature - love those homographs).

This is close to my heart because my first name is Zoë, and finding those damn dots in different bits of software is one of the banes of my life. 

My name is Greek, which means the dots are not umlaut, which is German, but dieresis. The dieresis tells the reader to 'pronounce these two adjacent vowels seperately', whereas the umlaut tells the reader how to pronounce the vowel marked (ä, ö, and ü are pronounced differently from a, o, and u.)

In terms of computer readability, that's not too much of a problem - it's a pronunciation issue, and (most) computers don't talk. English speakers are very lucky in that there are no non-alphabet characters in our written language that have the capacity to change word meaning, but I can't help but think that it can make for some lazy programming and difficult-to-use text editors. Arabic speaking programmers must tear their hair out when faced with written-for-English text editors. (As with Hebrew, writing in vowels in Arabic is optional, as the reader should be able to work out the word meaning by context - not handy for Arabic google.)

(Related: lots of people know that Hawai'i has only eight consonants, but did you know that across the Pacific, the consonants actually decrease as you move Westward? Polynesian languages (a subset of Austronesian language) are spread from NZ through the whole Pacific, all the way to Hawai'i. At every new island/language area from NZ West, you get a few consonants less, and they get softer, too. There's a nice map here: http://www.ecai.org/austronesiaweb/Maps/All-austronesia-area/All-austronesia-area.jpg)

Friday, 20 February 2009

the sweet darlin' conundrum

A little background:

I am a Fan of a very specific type of music: Northern Soul. 

(Actually, I am enough of a fan that I had to buy a second 'NS only' ipod, because my constant-loop NS playlists were making my husband insane.)

People have massive arguments about what Northern is, but I'm going to try for a definition (I hate to think what another fan would think of this, but anyhow:) 

-Northern Soul is 1960s American soul music that was not popular in America, but was popular in the North of England. Hence the name.
-Northern tracks are universally 'rare'. No well-known mainstream hit qualifies as Northern Soul. 
-Northern Soul is soul music you can dance to.

Having found & explored the surface of Northern, I now face a problem: FINDING MORE. 

The education geek in has recognised this as a problem with DISCOVERABILTY. While I haven't solved the problem, the elements that make it a problem are really interesting. 

The part that makes it interesting for me is that metadata (standard and user-generated) doesn't actually help. It pains me to say it, but it's true. 

The problems are as follows:

1. The dominant music technology of the time was the 'single'. Consequently, NS has always been identified on a track-by-track basis. Example: Marvin Gaye's 'it takes two' is NS (not a favouriteof mine, but still), Marvin Gaye's 'sexual healing' is not. By implication, NS tracks cannot be assessed on the basis of artist. 

2. And on that topic, track name isn't a definitive guide either - 'Stop (in the name of love)' by the Supremes is not NS, but 'Stop (in the name of love)' by the Flirtations - i.e. the much more enthusiastic original version - is. A more dramatic example: the 80s hit 'Tainted love' by Soft Cell was first recorded in 1964 by Gloria Jones. Guess what? It's an NS track (and much, much better in the original). So, NS tracks cannot be assessed on the basis of song title. 

3. NS is, consequently, assessed on a completely subjective basis. Comments on Mary Fox's amazingly cool radio show are frequently filled people saying 'THAT'S NOT NORTHERN SOUL!!' There is no objective approach to what constitutes NS.

4. A lot of NS still exists as singles. Rare singles. As in, not on itunes, but actually sitting in a box in someone's living room. The most famous (& imho best) NS track ever had - by rumour - only three original copies & is now the most expensive single ever sold. Thank gravy you can get it online. The body of content exists in a range of media, some of which are physical media.

5. NS uses a specific sub-genre terminology (i.e. a controlled vocabulary) that is not reflected in itunes etc. - search for 'floaters' or 'stompers' and you'll get nothing back. Social terminology has not translated into search terminology.

6. Many NS fans (and they are a passionate bunch) found NS in the 60s, and never - I mean never - looked back. NS is bizarre that way - it's a trend that just refused to die, and is still refusing 40 years later. Amazing. I love these people, so dedicated. But those folk already have their NS community, and for the most part, it's in the real world. Point being, the most valuable source of knowledge I can tap is not available in the social networks I access, i.e. online. (Having said that, the Mary Fox listeners are on the message boards all the time - big props to them.) 

And now, the big one:

7. Recommendation engines don't work. They don't work because the content scope is too specific and the algorithms just can't keep up. In pretty much any distribution environment (itunes, last fm etc) the recommendations will tend towards the middle ground. You find an NS track you like, and the recommendation engines will steer you into conventional territory - the Everly Brothers, the Supremes, pretty much anything from Motown records. They just can't manage the granularity.

It's easy to understand. I like the Everly Brothers. I also love 'Hold On' by the Radiants. Now let's imagine another NS fan, Suzy. She likes the Everly Brothers. She also loves 'Try a little harder' by the Fi-dels. Jason likes the Everly Brothers, and loves 'Too late' by Larry Williams and James Watson, and 'Sweet darlin'' by Jimmy Soul Clarke. All three of us have downloaded both the things we like and the things we love.

You see the common demoniator, yes? That's what the recommendation engine will steer us to.

Hence the problem with the track-by-track assessment of the genre, and the lack of 'artist' as a unifying metadata mechanism for discovery. The underlying algorithms are far too complex for me, but experience shows that in this domain, they aren't up to scratch... so far.

For now, I'm going to stick with listening to a lot of differnt internet radio shows and writing down the tracks I like on a piece of paper + hoping I can download them, which I usually can't. That'll have to do.

(And if you like soul, please hunt down the tracks above - they're fabulous.)

about digital comics

A wonderful exposition from balak01 at deviantart:

Tuesday, 17 February 2009

The cutest astronomy EVER


If I didn't have a blog, I would create one just to link to this. 

https://www.galaxyzoo.org/

In the galaxy zoo, members of the public can help astronomers by looking at pictures of far-flung galaxies and - so sweet! - tagging them.

Turns out that good old pattern recognition still requires human eyeballs. Computers just couldn't pull off this level of complexity, but an army of starloving office workers on their lunch break? Absolutely. Job done.

(For the sake of this post, I have chosen to define visual tags as metadata, so yes, it does count.)

Sunday, 15 February 2009

Powerset

I like it. It's got potential.

As described on ReadWriteWeb:

Powerset is a much-hyped semantic search engine that uses natural language processing to "understand" concepts in web content and match pages to queries. Right now it only searches Wikipedia (and Freebase). 

www.powerset.com

The string 'Semantic search' always makes my ears prick up. I had a play with powerset, and it is not a bad start in the whole Natural Language Processing search experience. (Incidentally, I am so sad that NLP, which is one of my Favourite Things, shares an acronym with corrupt, pseudo-scientific, fraudulent, thieving, and cultish self-help psycholbabble. Oh well.)

Powerset has a smashing feature for information discovery: Factz. A search for 'Gorilla' (NB: 'Gorillas' takes you to the band 'Gorillaz') gives us this: 

1. A short overview ('Gorillas are the largest of the living primates'...)
2. Scientific name, Taxon rank, etc. 
3. Factz. (This is the fun bit.)

The factz, each of which are linked to info tell us that...

Gorilla uses: tools, branches, lick, rocks, stick, saplings, and trunk
Gorilla eats: plants, fruits, shoots, bamboo, herbs, and leaves
Gorilla live in: forests, groups, corners, troops, and park.

Not bad. (And if you're wondering - Gorillas lick objects to identify them. Who knew? I didn't, hence me being impressed by the discoverability potential - you can learn by accident, which is my favourite way.)

It all gets a bit stickier with more complex topics - like people. A search for Marlon Brando opens with his bio, provides his Date of Birth, Date of Death, Place of Birth, Nationality... very impressive. But then we go to factz:

Marlon Brando played: Jor-El, role, owner, survivor, movie, Corleone, drums, characters,
Marlon Brando said something about: Kashfi, Tango, Paris, Brando, Last, Bertolucci, call, Marilyn Monroe
Marlon Brando portrayed: father, leader, Don Vito Corleone, Johnney Strabler, Jor-El 

So... what's the semantic difference between 'portrayed' and 'played'? And how did 'role' get through the net? (Answer: the sentence '...in the role eventually played by Marlon Brando'.)

But that's a quibble. I think the problem can be solve in the first instance by not paying heed to obviously off-target factz (like 'role'). And I love 'said something about' - it's a little misrepresentative because it is, by definition, limited to available quotes, but still - how nifty!

I hope this one takes off, it's terrific. If it does fly, the big danger is that it may be so good that people take the results as being comprehensive - let's all promise not to do that, shall we?

Black ice and collaboration

I ride my bike to work. Last week, that process was hampered by a huge fall of snow. The snow quickly turned, as snow tends to, into ice.

Ice and bikes do not make a good combination.

My route to work, across a field, became a death trap. There was one particular stretch, some sixty meters long, where getting across with ankles intact became impossible. The ice was thick and treacherous, and - worse - rutted with wheel tracks seemingly designed to send you flying.

Then, on the third day, something amazing happened.

There was a clear path, about twenty centimeters across, straight down the middle of the path, with ice either side. How?

Let's assume that each cyclist is acting in his/her own interest. That interest is 'not falling off'. They pick the clearest path. The clearest path is the one with least ice. Wheels create friction as they go, eroding the ice, meaning that the clearest path is the one taken most often by people who went before. 

So - each individual cyclist uses the clearest path, and by doing so, makes it clearer for the next person until the route is safe.

No-one consulted on this. No-one 'took one for the team'. No-one planned it. Everyone acted independetly in their own interest, and yet, the result was a collaborative solution to a difficult and, initially, intractable problem.

The analogy I'm reaching for here is Open metadata standards. I would argue that it is in the self interest of all cataloguers and indexers to use them - they're comprehensive, supported by a community, and (most importantly) they mean you don't have to reinvent the wheel. There's no need to break your brain creating a model that will accomodate aggregation when IEEE LOM provides one. That alone saves both time and money. 

And by following this self-interest, something collaborative happens that solves an intractable problem. The problem is interoperability, and the solution is standards. Any option that is not based in standards means hitting a wall every time the data has to be exploited in a new environment, which it always, always will be (even if it's an upgrade to internal/proprietary software). And if there's a seismic shift in technology, and a new standard is required, well, everyone will be in the same boat - and better solutions to data migration can be generated better, faster, and - compared to converting proprietary formats - much cheaper. That's the 'avoiding danger' part of the analogy - we all want to get through the data management ice patch without breaking our leg.

It's the one thing no-one ever thinks to say about open standards - they're good for business. For publishers, I don't think there's any excuse left not to use them. 

Saturday, 14 February 2009

Twitter

I have always been late onto bandwagons. It's a tradition with me. So I am proud to say that I have - finally - joined twitter.

I'm a newbie, and I'm only following twenty odd people, but already I have some questions regarding the structure of the feed flow.

I may be missing something, but is there a way to categorise feeds? If not, why not?

My feed contains mostly elearning types. It also contains Stephen Fry. Why? Becuase I love him. Also, he's the second most followed person after Barack Obama, and who am I to doubt the wisdom of the crowd?

Now I like face-to-face collaboration as much as online, and I'm looking forward to sharing my feed with colleagues. I plan to actually show them, i.e. with two of us looking at the same screen and clicking things. But do they need to know I follow Stephen Fry? Especially when he's not always temperate in his tweets? Also - what if I was following my actual friends? (They're not on the site yet, but they might be eventually.) I wouldn't want them all mixed up in conference announcements.

On the other hand, maybe I would. What a lovely melange, to have real life and work life in the same bubbly pot. But it would stop me sharing - at least with my colleagues - and isn't sharing the point of twitter? 

How strange, that a service that is only about sharing could, through it's own structure, stop me from sharing. 

I'm going to investigate twitter clients now - maybe I'll find a solution there. 

Wednesday, 11 February 2009

Google hacks

Discoverability - it's inescapable. Every time you use google, you are discovering. With that in mind, here is my favourite underutilised google hack: Search in site.

Say you're looking for an article about digital archivists that you read in the New York Times. You can try any number of strings, including string "new york times", and get nowhere. Try this instead:

digital archivist site:nytimes.com

Using the site: function, Google will only search www.nytimes.com. Neat, huh?

Tuesday, 3 February 2009

Ada Lovelace day

As documented at http://findingada.com/:

Ada Lovelace Day is an international day of blogging to draw attention to women excelling in technology.

All you need to do is sign the pledge, pick your tech heroine and then publish your blog post any time on Tuesday 24th March 2009. It doesn’t matter how new or old your blog is, what gender you are, what language you blog in, or what you normally blog about - everyone is invited.

To help us keep track of the conversation about Ada Lovelace Day ahead of the event, and to pull together everything on the day, please use these tags:

  • AdaLovelaceDay09 for Delicious, Technorati etc. It’s long, but clear.
  • #ALD09 for Twitter hashtags.

I already know who my tech heroine is, and while I'll write this again on the day, I'm happy to nominate... LUCILLE BALL!!

No, I'm not joking.

Lucille Ball is one of the great unrecognised media figures of our time. Media is a technology, and technology is increasingly a medium, so why not? Amazingnesses include:

1. Lucille Ball invented the repeat. And if you think that's a so-so achievement, get yourself a copy of 'Everything Bad is Good for You' and get yourself some learning about the power of repeatability. The more times a consumer can watch a show, the more potentially profitable it can be over time, and the greater incentive there is for producers to improve the quality. DVD box sets, people. And yes, I am saying Lucille Ball is responsible for the Sopranos.

2. Lucille Ball opened people's minds & broke down prejudices. Desi Arnaz (Ricky) was Cuban, people. And this was the 50s. (And yes they were actually married, but did divorce after 20 years or so.)

3. Lucille Ball was the first female head of a TV production company (DesiLu). 

4. Lucille Ball produced Star Trek.

HA! Bet you didn't know that last one, did you? Totally true, and she gets the credit for Mission: Impossible, too.