Twitter is the new messaging bus - very cool

In a recent blog post I said that I thought that Twitter was the new TCP/IP. What I meant by this is that Twitter is the messaging protocol which provides value to others, making it almost impossible to monetise in its own right.

To back this up, I've spent a few hours knocking up an example. I've built a new bot service which uses Twitter for its inputs and outputs.

GSearch lets you use Twitter to search the Guardian newspaper's archive. To use it, you need to follow 'gsearch'. A few seconds later, it will follow you back (and increase your twitter stock, if nothing else).



To perform a search, simply send a tweet with the hashtag #gsearch and the search terms you want to look for. The bot will read your tweet, take the search terms, search the Guardian's archive, compress the url of the first hit with Bit.ly, and send you a direct message in return. It checks for new messages once a minute, so you should get your message back quite quickly.

The point of this is to demonstate that Twitter is at the same time, enormously valuable yet impossible to monetise. At no point during this service do you have to visit Twitter's site.

This isn't a managed or supported service though - its just running on my PC. If it is off, then its off, but if its on, then I think that it is quite a nice demo of mashing up applications using Twitter as the messaging bus.

The upshot? Perhaps the easist, most compatible mobile interface to the Guardian's web site. Simply use your mobile twitter client of choice, and wait for your search result to come to you.

Let me know what you think

Lies, damned lies and statistics

Right, I'm not a great fan of the iPhone. This probably stems from my deep hatred of Apple. This posting isn't an anti-iPhone rant - I just wanted to disclose my preference in advance.

About 20 minutes ago, this post went up. It states that the iPhone is only the 24th most popular phone for mobile web browsing, with Nokia taking the bulk of the honours. The problem is that this data is enormously skewed.

The source in from Bango, the mobile payment processing folk. They've collected their statistics by phone model for February 2009 and they've reached this conclusion.

Except this isn't the correct conclusion. The conculsion is that iPhone users don't use Bango very much. Well, this isn't a shock, iPhone users only buy apps from the Apple App Store, they don't go hunting around. They aren't being lead by operator portals for instance.

Why am I angry about this? I don't know, other than bad maths from other bloggers I suppose.

Who should buy Twitter?

Much has been said about potential buyers for Twitter over the last quarter. Names have been dropped online as dead certs, and many different blogs give different reason as to why Corporation X should empty their wallets.

The Motley Fool recently published their take on why Google should buy Twitter. They've even predicted that it'll happen before the end of March 2009. Waaayy back in October 2007, SmoothSpan blog was urging RIM to get in and buy it before it got too expensive.

So, which player do I think has been discounted in all the predictions? Is it the same player to whom Twitter would represent the most value? I think so, and I think it is Microsoft.

Microsoft have poured billions into their online services, and are still a bit player. Almost all of their services have a more popular version elsewhere. The latest incarnation of Microsoft's online services, Microsoft Live is actually quite good, but its usage seems to lag behind competitors (notably Google) in almost every respect (figures are hard to obtain, but Hotmail might still outstrip GMail in numbers of accounts).

Twitter has a problem. No, not that one. Twitters problem isn't that it is overcrowded and the servers keep running out of capacity. Twitters problem isn't actually that it isn't making any money. Twitters problem is that it lies at the bottom of the food chain.

Look at the number of services which have launched since Twitter started, based off Twitter. And not just the client applications or search engines. Applications like twanswers are able to sit higher up in the food chain, and have a great opportunity to make money, but it is Twitter which is spending the money creating the enviroment/platform/ecosystem (delete according to preference) which makes the twanswers service work.

The same goes for Blellow. It provides a useful and simple function - Twitter Groups - helping you sort out the million tweets in your feed. Again, there is great potential for making money, without the expense of running the infrastructure (technical and social) behind it.

So look again at Microsoft. What Microsoft is lacking is critical mass. The apps are fine, the brand is well known, the uptake is rubbish. When was the last time you logged into Live, if ever? If Microsoft buy Twitter, then they are buying millions of Twitter users, and getting them to stick their noses into Microsoft online portfolio.

The traditional argument against Microsoft as a purchaser of Twitter is that Microsoft won't buy it without a plan for profitability. My argument is the complete reverse. Microsoft are the only people who will buy it without a plan for profitability. Twitter is the loss-leader which could kick-start Microsoft's portfolio back into life. Google won't buy it, despite what anyone else says* because they have got a plan for profitability for Twitter. There is only one way to actually monetise Twitter, and that is to inject adverts into your feeds. Google could do that day one. Twitter could do that today as well. The reason that Twitter haven't done that is because that signals the instant death of Twitter. Twitter isn't technically sophisticated. Actually, more or less any script kiddy could put together a Twitter clone in a few hours. If Twitter starts serving up ads in your feeds, someone else will do a better, cooler Twitter and everyone will jump ship in an instant.

Twitter needs to find a buyer, we can all see that. Microsoft are the only player with the financial clout, the need to buy the user-base, and are the only ones who will not destroy the service, but see it for what it is - the new TCP/IP. Twitter is the protocol which transfers data between applications. Ok - so susbsitute 'applications' for 'users', and 'data' for 'ideas' and this makes sense. Twitters future isn't necessarily sending stupid messages to your mates, it is a general communication process, like enterprise busses for the majority. But that is another blog for another day.

* I could look really stupid if you're reading this in the future, and Google have just announced that they've bought Twitter for $100 billion. If they have, stop using Twitter because it is about to become rubbish.

Building super fast, super portable mobile maps - Part Three - Making it super fast!

So, my maps are scrolling around my mobile screen really nicely. I'm jealous of how fast GMM is, and I can't see how they've got it so fast.

Jaak Leineste again had the solution, in this post

" We have solved it with the Mgmaps lib SDK and CloudMade tiles this way, that we set up own "tile streaming" server to AWS EC2, so it has superfast connection to CM tile server, and I am doing the tile proxying there just like you describe. I must admit that I got also some inspiration from the google's mobile maps app :). You can compare different map reading options, direct 64x64 tiles or the multi-tile requests (we call it "streamed") with on-line web runner demo at http://www.nutiteq.com/libsamples.html#BasicMapperapplication . Start the app from web (it starts microemulator automatically), you see performance with default map, which is CM 64-pixel individual tiles. Select Menu > Change map > select CloudMade streaming (via Nutiteq) and see speed then. Difference is quite big, to say the least. Not worse than with google maps mobile."

So this is the secret to super-fast mapping - you have a webservice which fetches tiles for you and concatenates them into a single download. You then pull them out of the stream. The reason the old method is slow isn't because of the speed of my connection on my mobile, its the latency in setting up the http request. If I've got fewer requests, then I've got a faster map.

My problem was that the MoSync image downloader doesn't provide me access to the stream. It just gives me a formatted PNG image I can display. I needed to write my own downloader.

MoSync does provide access to their connection object however. With this, I can have direct access to the download screen. So with the help of Niklas and Fredrik at Mobile Sorcery, I wrote my own downloader which would retrieve tiles out of a stream.

I wrote my own streaming service in C#. This takes my API key and the details of the tiles I want from CloudMade. It fetches them in order from the CloudMade server, and returns them back to the requestor, flushing the response stream after each tile so that the client application isn't waiting needlessly.

This has made a massive impact on my mapping component. Previously it would take 30-60 seconds to get all 36 tiles. Now I get all the tiles I need on the screen in < 10 seconds, and all 36 tiles in < 20 seconds. Personally, I think that my component is at least as fast as GMM.

Building super fast, super portable mobile maps - Part Two - Getting Maps onto the Screen

So, I've got a client who wants mobile maps, and I've got a supplier of map tiles. I've signed up as a developer on CloudMade, got my API key and I can see tiles. Now I've got to get them on screen.

About a 40 years ago (ok, about 25 years ago maybe) I was working out how to get my Acorn Electron to play Chess. I was just hacking around more or less directionlessly, and I fell into a classic programming trap.

My thought process was: "A chessboard. That's an 8x8 grid. So I'll represent the board in memory with a two-dimentional array. 10 DIM BOARD[8,8]". This seems obvious, but it is not the easiest way to implement a grid where you want things to move from one place to another place, and it isn't too quick in implementation. For example, to calculate valid knight's moves becomes fairly complex. If I've got a knight at [2,0] then I need to try permuations of where it can go in both dimensions. If I created a board as a single dimension array (10 DIM BOARD[64]), then I know that valid moves are:

currentLocation + 6, currentLocation + 10, currentLocation + 15, currentLocation + 17 currentLocation - 6, currentLocation -10, currentLocation - 15, currentLocation - 17

provided that the result is between 0 and 64.

Similar decisions are needed when you create a map grid on a mobile phone. So, again I started with an assumption of having a 240x320 screen. If I imagined a grid of 6x6 squares for the map tiles, then that will give me a total grid size of 384x384 - enough to cover the whole screen, plus a margin all around so that when the user scrolls, the tiles are ready. If I create this grid as a single object array[36] then it makes life really easy when it comes to scrolling. More on this later.

So I created a class to represent a MapTile, a marshaller to handle the downloads, and a Widget to put the map on screen. The widget converted the location from the GPS (or location API) into a CloudMade tile reference, and then took this as a basis for calculating the references for all the other tiles.

Each MapTile is responsible for formatting a URL which will get the image from the CloudMade server. It requests a download from the marshaller, and in turn gets its image.

The widget then puts each image on screen as they are downloaded, building the map. It also handles the scrolling, where the smart decision about an array MapTile[36] pay dividends.

To put the map on screen, I need to know which tile is going into the top-left corner. I call this the 'origin tile'. When my map starts this is the tile at MapTileArray[0]. The tile with the actual location I want to see is MapTileArray[14], which is about the middle of the screen. I start by displaying MapTileArray[originTile] top left, and then rip through the array in order, adding 1 MOD 36 to get to the next tile. This means when I get to MapTileArray[35] and want the next tile, I know that the next tile is MapTileArray[0].

I also wanted it to prioritise which tiles to get first. I don't want it to start with the top left tile and work sequentially though. I wanted it to start with the tile which shows the relevant location and spiral out. Initially, I worked on an overly complex algorithm which will calculate the order that the tiles should be downloaded in at runtime. In the end, I got some common-sense together, and just set up the order in code. I add each of these values to the value of the origin tile, MOD 36, and that is the order of the tiles that I want.

If I'd gone with a two dimensional array, then I've got an array I want to reflect on screen. This means that a scrolling operation doesn't mean adding to subtracting either 1 or 6 (depending on the direction of the scroll), but I've got to switch all the tiles into different locations into the array to maintain its reflection of the screen. Massive pain.

So I can scroll my map around nicely, my array is working quickly and easily. Its a lot slower than Google Maps though, and I start to wonder why.

Part Three >>

Building super fast, super portable mobile maps - Part One

I've spend the last couple of weeks building mapping into one of my projects. It is something I've resisted in the past as being too complicated, too difficult and not offering a lot of value. However, my client wants a location based messaging solution, so maps we're a must have.

My client (who shall remain nameless for now) said that it was easy. His previous partners had done it really quickly; just mash-up Google Maps. The problem is that this is a very web-oriented view of development. Whilst GM are easy to mash-up in a web environment, they aren't licenced for mobile application development. Their API is also Javascript-based, so I wasn't going to be able to use that either.

So the next stop is just to hack it. I found that many people have done this in the past. You just need to know how Google Maps for Mobile (GMM) converts co-ordinates into map tiles, and you can build URLs to get the correct tiles. I also discovered that everyone who had done this had quickly received 'cease-and-desist' letters from Google's lawyers.

So what are the alternatives? Yahoo! Maps are as rigidly licenced as Googles. Microsoft are much more relaxed about accessing their geodata, but they only make tiles for the web.

The issue here is the size of the tile. Let's assume for the sake of argument that the normal size of a mobile screen is 240x320 pixels. The tiles used on the websites are 256x256 pixels, which means that one tile will cover almost the whole screen. If you've only got a 128x128 screen, then you've got something which is way to big, giving the developer the additional headache of working out which bit of this tile needs to be on screen. The main problem here is that this is one download which means a big delay between hitting the button on the phone and seeing the map. GMM has mobile-friendly map tiles sized at 64x64 pixels. This means that I can convert my location to a single 64x64 tile, and centre my screen around this.

So a quick bit of research later brought me to the OpenStreetMaps initiative. This is crowd-sourced geo-data. You go out with your GPS and send data about all the streets in your neighbourhood. This becomes 'open sourced' data. Digging a bit more lead me to CloudMade, the extension of OSM to convert their raw data into useable map tiles.

CloudMade have fantastic support for mobile. Not only do they do their tiles in 256x256 and 64x64 pixel tiles, but they also do their mobile tiles with high-contrast colour schemes. Now they are extending this, so anyone can create custom cartography with their style editor.

Better news seemed to be that Jaak Laineste has developed a J2ME component from his compay Nutiteq. This wraps up all the functionality you need to make your own GMM-clone with maps from CloudMade. This would be a massive short-cut, except that I'm not writing in J2ME; I developing in MoSync.

<digression>

The number one problem with mobile development is that every damn phone is a little bit different. The purpose of Java is that you write your code once, and it will run in an identical fashion on a different platform. The problem with this is that it has never quite been true, and for the Java you've got on your phone, this problem is magnified by a thousand times.

The problem is that you've got one body specifiying the language, and dozens of bodies implementing it. Just as no two people will interpret a book in the same way, no two implementers make J2ME work in exactly the same way.

The upshot of this is that even when you've got your J2ME application working on your phone, there is no guarantee that it will run on anyone else's. Or a Windows Mobile device, or in an S60 environment.


The guys at Mobile Sorcery have taken this pain away. They've written their own API which they've tested on the handsets. The difference with this approach is that it is one body who is managing implementation, massively improving compatibility. You write (or port) your code in C/C++ against their API, and it will build your application for hundreds of phones. Each of your target handsets gets a build specifically for it. On J2ME phones, it builds a J2ME application. On Windows Mobile devices, it builds a Windows application. On S60 devices it builds a Symbian application.

</digression>

Still, even without being able to use Nutiteq's mapping component, I've got a supplier of map tiles.

Part Two >>>

The Three Roadblocks to Mobile Web

I seem to be giving this talk a lot lately, so I thought I'd pull my finger out and blog something.

Many handsets have had web connections (and actual http/html connections, not just wap) for many years. On some devices its even quite good. But it has never made the penetration into normob psyche that many people predicted. I'm often asked why this is, and why I continue to evangelise on-device portals and applications over mobile web given in availablity of all-you-can-eat data plans.

To illustrate my point, I distilled my thoughts down into what I call 'The Three Roadblocks to Mobile Web'.

So you've built a brilliant mobile web site. Perhaps you've used one of the automated tools to rebuild your main site and represent it to a mobile user. The problem is, you're not getting any hits. What is the problem? Does your content suck? You are getting loads of traffic through traditional routes, but not through mobile. How can you get more people to your site?


1. Having the idea

The number 1 problem with mobile websites, is to get people to have the idea to use their phone to do some web surfer at a convenient moment. I used to use the example of someone waiting on the platform for their train, but Paul Golding recently pointed out to me that the peak time for mobile web use is between 23:00 and 00:00 - when people are in bed. Anyway, you can imagine your ideal user who is idling for a few minutes - now is the perfect time for them to visit your website. The problem here is that even when people are in this situation, even when they're playing with their phone, you've still got to get them to press that Internet button.

There are a lot of people who are scared of this button. There persists an idea that connecting to the web is expensive and unpredictable. Even my wife, who I consider do be reasonably technical, occasionally asks me how long I've been reading that article, like it billed on connection time.


Having an on-device application puts your content in view. If your content has been downloaded and cached to be read offline, then people are much more confortable about starting the app. The fear that you don't know how much it is going to cost to click the next link disappears, and your content icon is one of maybe a dozen application icons on most phones, and not one of a million billion sites.

2. Knowing you exist

So, even if you've got someone in bed doing a bit of late night surfing. How do you get them to browse to your site? Mobile traffic isn't driven in the same way as desktop traffic. You can have all the SEO and tagging and cool tricks in the world, but the fact remains that typing a couple of keywords into that Google box you've got at the top of your browser is very quick and easy, and typing even one keyword into the Google box on your phone is more trouble than you can be bothered with, even if your really interested in the result.

3. Getting to your site

Traffic is driven from operator portals. Your late night surfer is going to see a couple of things on their home screen that catch their eye and click through on those. They're unlikely to go off that. If they can't be bother to search Google from their phone, they're definately not going to type your URL in.

So you spend some money, and you get a keyword on a shortcode. You advertise it, and the auto-responder sends your URL back. And some people even save it. In the list of bookmarks in their browser. Out of sight, out of mind.

So this is why I back on-device applications.

1. It is visible - you've put your icon and your brand literally into the pockets of your readers/customers
2. It is memorable - you're application is now in direct competition for eye-time with your user's browser. You're way up the decision making tree now.
3. It is comfortable. I build ODPs to download and cache data in a controlled way. This isn't necessarily any less data, but it is more predictable for the user. They feel in control, and with off-line browsing there is no fear in pressing the next link.

And if you're thinking of building an app store, then why are you thinking of building it as a website? Put your icon on your customers phone.

I buy a lot of games from GameMobile, but I wish that they'd build their own app so I can browse the latest games comfortably from my phone, rather than on line. I promise you that I for one will be hitting that 'Buy Now' button far more often.