Fred Wilson wrote about the value of blogging and building social capital, demonstrated by the hundred requests for invites he received on his post on his recent investment, Boxee, an invite-only service.

Now, while I find the behavior of public invite-requesting curious, I understand it.

I also think there’s another side to this equation that I’d like to point out, being one of the fortunate early adopters who happens to get invited to a lot of early alphas and betas… and that’s understanding the relationship between the creator of the beta and the testers. Or, to put it another way, requesting an invite to a service for one’s own benefit is one thing; understanding that an invite is a privilege given in exchange for feedback and suggestions provided is another. And the secret to getting early access to beta programs is, perhaps obviously, to be a good beta tester.

There are any number of ways to demonstrate that you’re worthy of an invite to an invite-only alpha or beta program. One problem is that a lot of beta feedback is submitted privately, outside of public forums. Whenever I can, I attempt to use more public forums, both for my own recollection, but also for the benefit or other testers, developers and later users.

In other cases, I’ll use Flickr or Twitter, leading to interesting phenomena, similar to what Fred describes.

In particular, I’ve been alpha testing a music player called Spotify for some time. It’s an incredible service and recently opened up with three levels of service, although it’s sadly not available in the US yet owing to licensing issues. Now, the only way to get an account with the service is to request an invitation.

It just so happens that I screenshotted an element of the new interface, uploaded it to Flickr and titled the photo “Spotify Invites“. That photo is now the second result for that phrase on Google and people have noticed, quickly exhausting my supply of invites.

The problem with this scenario, and with Fred’s, is that many folks seem eager to get access solely for their own benefit, without thought to the quid pro quo that makes beta programs successful (and ultimately benefit both the developer and subsequent users!). And I think it’s worth it to point out that beta programs aren’t just freebie give-aways: the gate is there for a reason!

I wrote this post in 2005, back when Gmail was an invite-only service (!!) and I was thinking about the relationships we were attempting to cultivate with the Flock alpha tester program:

So what of all these invite-only (or formally invite-only) services where you have to know someone on the inside to get a golden ticket? Does it artificially increase desire? Does it help services grow organically and cut down on trolls and spam, creating more value for invitees? Does it create more investment from the user community and perhaps establish even minor connections between invitor and invitee? Or does it create a false hierarchy around an inner circle of well-connected geeks?

Who knows?

What I do know is that it’s a curious trend and happening rather profusely across the web. Good or bad? I can’t quite say — except that in the case of Flock, we’re using the invite system to start out slowly on purpose. We want to not only be able to scale up organically, but we also want to cultivate relationships with our brave early adopters so that we can build the best experience possible over time. And to that end — we want to make sure that when we do launch publicly, we’ve hammered out all the glaring issues — as well as minor ones — so that sum total Flock makes you more productive, more explorative, and more voraciously social on the web. So for now, Flock will remain available to few kindred souls with enough courage to shove through our bugs and dodge the sharp edges. In the meantime, do add yourself to our invite lottery so that your name will be there when the next round of invites go out.

Not much has changed in terms of the structure of invite-only betas (even though the tools for managing them have improved), but I think something of the intimacy and purpose of these programs have been missed as more of the mainstream have gotten used to handing out just their email address for access to such initiatives.

As Fred points out that there’s value in building up social capital so that you can help stoke interest in new projects and draw the interest of potentially valuable contributors and testers, but it’s just as important to highlight the value of diligent and hard-working testers who have an interest in improving products and becoming partners in the potential success of such projects. I think there’s the potential for mutually reinforcing and ongoing relationships in the execution of a productive beta program, and that those longer-term relationships should not be overlooked.

. . .

To that end, I’m looking for some highly motivated and qualified testers for LittleSnapper, Real Mac Software’s new webpage screenshot utility. Be one of the first ten to leave a comment with your proper email address and a description of how you approach beta testing and I’ll send you info on where you can sign up. As I’m eager to see LittleSnapper mature, I won’t settle for just anyone — prove to me that you’d add value to the alpha tester program!

Even though I wasn’t able to attend the eighth Internet Identity Workshop this week in Mountain View (check out the latest episode of TheSocialWeb.tv for a glimpse), I wanted to do my part to contribute so I’m sharing the results of a study that Brynn Evans and I performed on Mechanical Turk a short while ago.

I’ll cut to the chase and then go into some background detail.

Of the 302 responses we received, we only rejected one, leaving us with 301 valid data points to work with. Of those 301:

• 19.3% had heard of OpenID (58 people)
• 9.0% knew what OpenID was used for (27) and 8.0% were unsure (24)
• 1.3% used OpenID (4) and 18.3% were unsure if they used it (55).
• 5.3% recognized the OpenID icon (16) and 7.0% were unsure (21).

Combining some of the results, we found that:

• of those who know what OpenID is, 14.81% use it.
• of those who have merely heard of it, 6.9% use it.

That’s what the data show.

Background

Several weeks ago, Yahoo released usability research and best practices for OpenID (PDF). This research was performed by Beverly Freeman in the Yahoo! Customer Insights division in July of this year and involved 9 female Yahoo! users age 32-39 with self-declared medium-to-high level of Internet savvy.

This research, along with Eric Sachs’ later contributions (Google), have taken us from virtually zero research on the usability of OpenID to having a much more robust pool of information to pull from. And though I’m sure many would agree that this research only points to opportunities for improvement, many people interpreted the results as an indication that “OpenID is too confusing” or that it “befuddles users“.

A lot of people also took cheap shots, using the Yahoo! results to bolster their long-held arguments against the protocol and its unfamiliar interaction flow. The problem with such criticism, as far as I’m concerned, is that generalizing from the experiences of nine female Yahoo! users in their thirties is not necessarily representative of the web at large, nor are the conditions favorable to such research. Y’know, Ford got a lot of flack too when he introduced the Model T because everyone loved their horse and carriages. Good thing Ford was right.

Now, some of the criticism of OpenID is valid, especially if it can be turned into productive outcomes, like making OpenID easier to use, or less awkward.

And it serves no one’s interests to make grandiose claims on the basis of minimal data, so given Brynn’s work using Mechanical Turk (with Ed Chi from PARC), I thought I’d ask her to help me set up a study to discover just what awareness of OpenID might be among a wider segment of the population, especially with Japanese awareness of OpenID topping out around 28% (with usage of OpenID at 15%, more than ten times what we saw with Turkers).

Mechanical Turk Demographics

First, it’s important to point out something about Turker demographics. Because Turkers must have either a US bank account or be willing to be paid in Amazon gift certificates, the quality of participants you get (especially if you design your HIT well) will actually be pretty good (compared with, say, a blog-based survey). Now, Mechanical Turk actually has rules against asking for demographic or personally identifying information, but some information has been gathered by Panos Ipeirotis to shed some light on who the Turkers are and why they participate. I’ll leave the bulk of the analysis up to him, but it’s worth noting that a survey put out on Mechanical Turk about OpenID will likely hit a fairly average segment of the internet-using population (or at least one that doesn’t differ greatly from college undergraduates).

Method

Over the course of a week (October 19 - 26), we fielded 302 responses to our survey, paying $0.02 for each valid reply (yes, we were essentially asking people for their “two cents”). We only rejected one response out of the batch, leaving us with 301 valid data points at a whooping cost of $6.02.

Findings

As I reported above, contrary to the 0% awareness demonstrated in the Yahoo! study of nine participants, we found that nearly 20% of respondents had at least heard of OpenID, though a much smaller percentage (1.3%) actually used it (or at least were consciously aware of using it — nearly everyone (18%) who’d heard of OpenID didn’t know if they used it or not).

There was also at least some familiarity with the OpenID logo/icon (5.3%).

What’s also interesting is that many respondents, upon hearing about “OpenID”, expressed an interest in finding out more: “What is it? LOL.”; “I’ve gotta look it up!”; “This survey has sparked my interest”; “Heading to Google to find out”. I can’t say that this shows clear interest in the concept, but at least some folks showed a curious disposition, as such:

How can I tell for sure whether I’ve used OpenID or not when I don’t know what it is? I’ve surely heard of it. That confuses me mainly in Magnolia {bookmarking service} where I want to sign up, but I can’t as it asks for OpenID. And until you mentioned above, it simply didn’t occur to me to just search it up. Hell, after submitting this hit, I’m going to do that first and foremost. Anyways, thanks a lot for indirectly suggesting a move!!!

Now, I won’t repeat the other findings, as they’ve already been reported above.

Thoughts and next steps

The results of this survey are interesting to me, but not unexpected. They’re not reassuring either, and they tell me that we’re doing well considering that we’ve only just begun.

Consider that 20% of a random sampling of 300 people on the internet had at least heard of OpenID, before Google, MySpace or Microsoft turned on their support for the protocol (MySpace announced their intention to support OpenID in July).

Consider that nearly a year ago Marshall Kirkpatrick sounded the deathknell of what seemed like the forgone conclusion about OpenID:

Big Players are Dragging Their Feet … Sharing User Info is a Whole Other Matter … Public Facing Profiles are Anemic … Ease of Use and Marketing Clarity Remain Low Priorities

Consider that no concerted effort has been made to date to inform or educate the general web population about OpenID, or about the problems with sharing your user credentials all over the web, and that many of the large providers have yet to turn on their OpenID support (despite all coming to the table and agreeing that it’s the way forward for identity on the web (save, as usual, Facebook, looking more Microsoftian by the day).

Consider also that momentum to rev the protocol to accommodate email addresses in OpenID is just now gaining traction.

In other words, with areas of user education becoming obvious, with provider adoption starting to happen (vis-a-via MySpace demonstrating the value and prevalence of URL-based identifiers) and necessary usability improvements starting to take shape (both in terms of the OpenID and OAuth flows being combined, and in terms of email addresses becoming valid in OpenID flows), we’re truly just getting started with making OpenID ready for mainstream audiences. It’s been a hard slog so far, and it’s bound to continue to be challenging, but the shared vision for where we’re going gets clearer every time there’s an Internet Identity Workshop.

I plan to re-run this study every 3-6 months from this point forward to keep track of our progress. I hope that these numbers will shed some much-needed balanced light on the subject of OpenID awareness and adoption — both to demonstrate how far we have to go, and how far we’ve come.

While the news that Google is now an OpenID Provider was generally welcomed, a common chorus decrying their support (along with others large OPs like Yahoo, Microsoft and others) at best as half-hearted, at worst as ruining OpenID has revealed a significant barrier to such large providers becoming relying parties (even beyond usability).

Eric Sachs (Google Security Team) writes:

One other question that a lot of people asked yesterday is when a large provider like Google will become a relying party. There is one big problem that stands in the way of doing that, but fortunately it is more of a technology problem than a usability issue. That problem is that rich-client apps (desktop apps and mobile apps) are hard-coded to ask a user for their username and password. As an example, all Google rich-client apps would break if we supported federated login for our consumer users, and in fact they do break for the large number of our enterprise E-mail outsourcing customers who run their own identity provider, and for which Google is a relying party today. This problem with rich-client apps also affects other sites like Plaxo who are already relying parties.

Fortunately there is a solution, and it was developed specifically because Ma.gnolia ran into this problem when it became an OpenID relying party. The result, nine months in the making, was OAuth. Eric even recognizes this:

We need standard open-source components on as many platforms as possible to enable those rich-client apps to support OAuth. That includes a lot more platforms then just Windows and Mac. The harder part is mobile devices (Blackberry, Symbian, Windows Mobile, iPhone, and yes even Android), and other Internet connected devices like Tivos, Apple TVs, Playstations, etc. that have rich-client apps that ask users for their passwords to access services like Youtube, Google photos, etc. If we build these components, they will be useful not only to Google, but also to any other relying parties which have rich-client apps or exposes APIs, and it will also help enterprise SaaS vendors like Salesforce.

As I’ve been thinking about this problem, I’ve come to see as an intermediate approach to full-on delegated authorization a simpler, perhaps more familiar approach that would be relatively easy to implement given common interface patterns today. For comparison, Pownce’s iPhone app originally used out-of-band browser-based authentication, leading to a swarm of user criticism resulting in a compromised solution that required embedding a web browser in the app. Less than ideal.

In my proposal, rather than ask for a user’s password, an easier-to-remember OP-issued numerical PIN would be used to authenticate requests. Better is that this approach is already supported in OAuth, it’s just not widely used yet (though is similar to how Flickr authorizes mobile clients).

The basic concept is that you’d have one password (or other strong authentication method) for your primary OpenID account and you’d have one (or more) PINs that you would use to access your account remotely — perhaps in limited risk scenarios or where (again) the full browser-based OAuth flow is not possible or warranted.

Although I initially opposed FriendFeed’s use of Remote Keys, I now think that there’s some merit to this approach, as long as the underlying mechanism uses standard OAuth calls.

There are plenty of holes in this approach, but insomuch as it enables an existing pattern to be phased out gently, I think it offers at least the foundation of an idea that could be useful. It also could be used as a counter-balance to some of the current thinking on federated login flows with OAuth.

Consider these three sign in boxes for comparison:

1. Traditional Password
   
2. Lightweight PIN access
   
3. Full OAuth
   

Thoughts welcome.

Julie Zhou of Facebook discusses usability findings from Facebook Connect. Photo © John McCrea. All rights reserved.

Monday last week marked the first ever OpenID UX Summit at Yahoo! in Sunnyvale with over 40 in attendance. Representatives came from MySpace, Facebook, Google, Yahoo!, Vidoop, Janrain, Six Apart, AOL, Chimp, Magnolia, Microsoft, Plaxo, Netmesh, Internet 2 and Liberty Alliance to debate and discuss how best to make implementations of the protocol easier to use and more familiar.

John McCrea covered the significance of the summit on TechCrunchIT (and recognized Facebook’s welcomed participation) and has a good overall summary on his blog.

While the summit was a long-overdue step towards addressing the clear usability issues directly inhibiting the spread of OpenID, there are four additional areas that I think need more attention. I’ll address each separately.

Make it easier!

Overwhelmingly criticism of OpenID has been leveraged by developers and web users alike against OpenID’s ease of use.

For developers, implementing OpenID is confusing and cumbersome, and often tacked on as an afterthought to appease annoying early adopters (like me) who badger them to support the protocol. Even those who support the protocol report little upside, compared with something like Facebook Connect, which brings with it richer aspects of someone’s profile and social graph.

For web users, OpenID is confusing and frustrating, resulting in what I call “OpenID double registration taxation” — where a user, immediately following OpenID authentication, is prompted by the relying party (RP) to supply, and then verify, their email address. Why bother with OpenID if they’re going to have to go through the old school registration process anyway? Where’s the benefit in that?

On this latter point, we probably won’t make much headway until email harvesting goes out of vogue, which won’t happen until there’s a better way for sites to spam/bacn their members (bacn: “email you want, just not right now”), or until OpenID Providers (OPs) more consistently pass on profile attributes via SREG, Attribute Exchange or PoCo (or until people realize that email is dead to the MySpace generation).

Unfortunately, mandating that providers pass on profile data is something that cannot, and probably should not, be mandated by the OpenID spec, even though in comparison, Facebook Connect always provides some data. Fortunately OPs like Yahoo! are starting to improve this situation, by enabling opt-in controls that enable users to share their data more easily. If this trend continues, we may see fewer “double taxation registrations” and smoother OpenID login flows.

Still, for both end users and developers, OpenID must become easier to use and more obvious to implement. Fortunately, there is now fairly widespread recognition within the OpenID community of specific issues and a strong willingness to address them.

To that end, for example, advocacy for email addresses to be used as OpenIDs is growing, providing web users the convenience of reusing a familiar identifier, and affording developers a way to “upgrade” legacy userbases that may have been keyed to unique email addresses.

It is my opinion that enabling an email address to be used as a “hint” that resolves to a valid OpenID URL is a necessary step to dislodge one of the main nettles against OpenID. I also believe that this step is necessary to bridge the impending generation gap that’s sure to develop when MySpace flips the switch on their OpenID provider, enabling over a hundred million URL-based OpenIDs. Privacy concerns notwithstanding (remember, most RPs already demand a verified email address anyway), there are few reasons not to use email addresses for OpenID. I’d rather just make it so and let people pick for themselves how they feel most comfortable identifying themselves on services and move on to meatier issues.

Branding and marketing

On that note, Max Engel from MySpace brought up some important points about what it would mean to enable email addresses as OpenIDs. Soon to be one of the largest providers of URL-based OpenIDs (i.e. myspace.com/factoryjoe), he’s concerned that people will only implement support for email addresses if the OpenID spec provides a way to translate email addresses into URLs. This is a valid concern, but one that can be mitigated both in the language of the spec, and in the libraries that perform OpenID authentication.

Here is where I see an opportunity to finally establish OpenID as a brand unto itself, where the word “OpenID” can and should come to mean something to people (though of course not without an ongoing substantial and sustained marketing effort, lead by the OIDF, but primarily prosecuted through grassroots and community “spreading vectors“).

Here’s why: people have learned, over time, that “email” is easier to say (and shorter to type) than “electronic mail”. When you ask someone for their “email address”, most people on the web can give you the answer you’re looking for. We’re a long way off from the same kind of familiarity with OpenID, but ultimately you have to start somewhere. And because “URL-based identifier”, “blog address”, “profile link”, “home site” — ad infinitum — probably don’t mean much to anyone (let alone the same thing) there’s an opportunity to converge on a term that’s easy to say and captures the concept fairly well (or well enough) and is otherwise not known.

It’s also important to consider that not all URLs are in fact OpenID-enabled. This point alone is enough to convince me of the importance of the OpenID name and the potential for the brand. When you ask someone to sign in to your site, you can be pretty sure they’ll know what their email address is. If you ask them for a URL, and they provide you with a perfectly valid address but one that is not OpenID-enabled, they will not be able to sign in. If we can make it clear that “having an OpenID” is something special, and that not all URLs are OpenIDs, then we can begin to create the kind of awareness necessarily to confidently ask people for an OpenID, and have them respond appropriately.

It is here that I disagree with Scott Kveton, who has long argued that his mom didn’t “get SMTP, they got email”. I appreciate his sentiment and used to agree with his argument in principle, but now that I’ve thought about the fact that only “special URLs” are OpenIDs, I think it’s worthwhile to give that class of URLs a specific name.

Consistency

Furthermore, one of the greatest threats to the viability of OpenID is an inconsistent user experience. Unfortunately, this manifests itself both when signing in to a malfunctioning relying party, or attempting OpenID authentication using an OP that an RP doesn’t support (e.g. Microsoft Health Vault currently supports three OPs).

Another manifestation of this problem is that OPs are not required to consume OpenIDs. Though there’s validity in this complaint, change should not be forced at the technical level, because it really should be up to each individual provider to determine whose credentials it’s willing to accept. Now that the majors (save Facebook) have all gotten into the OP game (most recently Microsoft), it really just seems a matter of politics and inertia that none have moved to accept the OpenIDs of their competitors in any significant way (that is, neither Yahoo, Google, or Microsoft allow authenticating against their respective services using one of the other’s OpenIDs — and no, Blogger doesn’t count and Google hasn’t really released their OP yet).

While I’m sympathetic to Allen Tom’s argument that more OPs is frankly better for the web, I’m not convinced that a Visa card is all that useful if none of the major department stores will accept it.

I certainly respect large providers’ desires to both minimize the potential for abuse and to wade through the legal morass around identity technologies, but I can’t see how becoming an OpenID relying party is any worse than letting people create accounts with arbitrary (and untrusted) email addresses.

Hopefully through both political pressure and success-in-the-wild over time, we will see the majors become relying parties to their competitors’ OpenIDs for accessing accounts, and over a longer period of time, enable the use of personal/private OpenID providers or delegated OpenIDs (e.g. factoryjoe.com).

Should we see this situation change, I think it’ll bring about a watershed migration to patterns established by the majors — leading to consistency in the OpenID sign up and sign in experiences, and consistency in what people expect of OpenID account federation, leading to increased credibility and use of OpenID generally.

Leadership

But let’s get real: all these issues are going to require, above all else, solid foresight and leadership and a commitment to pushing through the thorny political issues that can often scuttle the best intentioned technologies (consider HD-DVD and Blu-Ray).

For reasons beyond my grasp, the OpenID Foundation has not met up to my expectations of leadership. Despite considerable progress in some areas, large swathes of stagnation have come to subsume many of the organization’s initiatives. International progress, as overseen by the OIDF, is lacking, except where local chapters (such as in Japan and in some European cities) have taken matters into their own hands. Code improvements to the OpenID libraries has languished and implementation of OpenID in various platforms and open source projects seems non-existent. Marketing simply isn’t happening and even if it were, I’m not convinced that there’s consensus on what we should market. And only now, after research from Yahoo and Google confirm what many critics have said for a long time is there finally work being done to address OpenID’s usability pitfalls.

Now, I realize that technologists don’t always make the best politicians (or designers or marketers for that matter) but that we haven’t seen the kind of OpenID visibility, credibility, innovation and adoption in North America that has been seen in Japan suggests to me that we’re either on the wrong course, or no apparent course at all. Worse, I fear that certain companies are already dividing up the proverbial “identity pie” before the damn thing’s even been put into the oven — a situation that needs to be addressed immediately by prioritizing a series of steps that the OIDF will take to establish OpenID in the marketplace, set firm how it will support individuals and companies alike, plot out its administrative and advocacy agenda for 2009, make clear its budgetary outlook, and list the marketing, design, education and research initiatives it plans for the coming year.

Without a clear path forward, I think that a lot of otherwise positive energy will devolve into useless sniping and infighting. Without strong leadership, we risk marginalizing many of the gains we’ve made to date in establishing OpenID as a core building block of the open social web.

For comparison, consider the progress that has been made with OpenSocial: only a year ago, people dismissed it as a “Gadgets API” (which, arguably it was). Since then, a large coalition of the willing has gathered to support and develop the protocol (which is still far from perfect, but demonstrates steady progress towards a goal), even convincing that old salt David Recordon that what they’re doing is decent. When OpenSocial 1.0 is released (they’re at 0.8.1 right now), there will be a distributed social graph with over 350 million potential customers available to developers (compared with around 100 million on Facebook). While David is right to point out, with Microsoft coming on board, there’ll be well beyond half a billion OpenIDs in the wild, that doesn’t mean that our work is finished. Rather, it’s just begun, and David sums up our situation fairly well:

While this is great news from Microsoft, real web-scale adoption of technologies always faces a chicken-and-egg problem between developers and vendors. Developers don’t want to adopt a technology without buy-in from platform providers and platform providers don’t want to support a technology if developers won’t use it. We’ve largely been able to successfully avoid this concern with OpenID as it grew from roots in an open source community with lots of people and companies involved in making OpenID what it is today. There are now well beyond half a billion OpenIDs available on the web which means we can mark the first phase of OpenID adoption, platform support, as a success.

The next phase of developer adoption will not be measured in the number of OpenIDs or sites that support it, but rather user experience, accessibility, and seamlessness of integration into a wide variety of applications and experiences.

To that end, there will be an Internet Identity Workshop in Mountain View November 11-12 where many of the primary participants in the ongoing identity conversations will converge. Historically the event has been one of the most productive in the space and with all the recent OpenID news lately, I’m hopeful that many of the issues I’ve mentioned above will be addressed and progress will continue to be made.

I will continue to be a staunch advocate of OpenID and think that it’s best times are still to come, but not without a redoubling of focused effort around concrete and ambitious goals.

Julie Zhou of Facebook discusses usability findings from Facebook Connect. Photo © John McCrea. All rights reserved.

Monday last week marked the first ever OpenID UX Summit at Yahoo! in Sunnyvale with over 40 in attendance. Representatives came from MySpace, Facebook, Google, Yahoo!, Vidoop, Janrain, Six Apart, AOL, Chimp, Magnolia, Microsoft, Plaxo, Netmesh, Internet 2 and Liberty Alliance to debate and discuss how best to make implementations of the protocol easier to use and more familiar.

John McCrea covered the significance of the summit on TechCrunchIT (and recognized Facebook’s welcomed participation) and has a good overall summary on his blog.

While the summit was a long-overdue step towards addressing the clear usability issues directly inhibiting the spread of OpenID, there are four additional areas that I think need more attention. I’ll address each separately.

Make it easier!

Overwhelmingly criticism of OpenID has been leveraged by developers and web users alike against OpenID’s ease of use.

For developers, implementing OpenID is confusing and cumbersome, and often tacked on as an afterthought to appease annoying early adopters (like me) who badger them to support the protocol. Even those who support the protocol report little upside, compared with something like Facebook Connect, which brings with it richer aspects of someone’s profile and social graph.

For web users, OpenID is confusing and frustrating, resulting in what I call “OpenID double registration taxation” — where a user, immediately following OpenID authentication, is prompted by the relying party (RP) to supply, and then verify, their email address. Why bother with OpenID if they’re going to have to go through the old school registration process anyway? Where’s the benefit in that?

On this latter point, we probably won’t make much headway until email harvesting goes out of vogue, which won’t happen until there’s a better way for sites to spam/bacn their members (bacn: “email you want, just not right now”), or until OpenID Providers (OPs) more consistently pass on profile attributes via SREG, Attribute Exchange or PoCo (or until people realize that email is dead to the MySpace generation).

Unfortunately, mandating that providers pass on profile data is something that cannot, and probably should not, be mandated by the OpenID spec, even though in comparison, Facebook Connect always provides some data. Fortunately OPs like Yahoo! are starting to improve this situation, by enabling opt-in controls that enable users to share their data more easily. If this trend continues, we may see fewer “double taxation registrations” and smoother OpenID login flows.

Still, for both end users and developers, OpenID must become easier to use and more obvious to implement. Fortunately, there is now fairly widespread recognition within the OpenID community of specific issues and a strong willingness to address them.

To that end, for example, advocacy for email addresses to be used as OpenIDs is growing, providing web users the convenience of reusing a familiar identifier, and affording developers a way to “upgrade” legacy userbases that may have been keyed to unique email addresses.

It is my opinion that enabling an email address to be used as a “hint” that resolves to a valid OpenID URL is a necessary step to dislodge one of the main nettles against OpenID. I also believe that this step is necessary to bridge the impending generation gap that’s sure to develop when MySpace flips the switch on their OpenID provider, enabling over a hundred million URL-based OpenIDs. Privacy concerns notwithstanding (remember, most RPs already demand a verified email address anyway), there are few reasons not to use email addresses for OpenID. I’d rather just make it so and let people pick for themselves how they feel most comfortable identifying themselves on services and move on to meatier issues.

Branding and marketing

On that note, Max Engel from MySpace brought up some important points about what it would mean to enable email addresses as OpenIDs. Soon to be one of the largest providers of URL-based OpenIDs (i.e. myspace.com/factoryjoe), he’s concerned that people will only implement support for email addresses if the OpenID spec provides a way to translate email addresses into URLs. This is a valid concern, but one that can be mitigated both in the language of the spec, and in the libraries that perform OpenID authentication.

Here is where I see an opportunity to finally establish OpenID as a brand unto itself, where the word “OpenID” can and should come to mean something to people (though of course not without an ongoing substantial and sustained marketing effort, lead by the OIDF, but primarily prosecuted through grassroots and community “spreading vectors“).

Here’s why: people have learned, over time, that “email” is easier to say (and shorter to type) than “electronic mail”. When you ask someone for their “email address”, most people on the web can give you the answer you’re looking for. We’re a long way off from the same kind of familiarity with OpenID, but ultimately you have to start somewhere. And because “URL-based identifier”, “blog address”, “profile link”, “home site” — ad infinitum — probably don’t mean much to anyone (let alone the same thing) there’s an opportunity to converge on a term that’s easy to say and captures the concept fairly well (or well enough) and is otherwise not known.

It’s also important to consider that not all URLs are in fact OpenID-enabled. This point alone is enough to convince me of the importance of the OpenID name and the potential for the brand. When you ask someone to sign in to your site, you can be pretty sure they’ll know what their email address is. If you ask them for a URL, and they provide you with a perfectly valid address but one that is not OpenID-enabled, they will not be able to sign in. If we can make it clear that “having an OpenID” is something special, and that not all URLs are OpenIDs, then we can begin to create the kind of awareness necessarily to confidently ask people for an OpenID, and have them respond appropriately.

It is here that I disagree with Scott Kveton, who has long argued that his mom didn’t “get SMTP, they got email”. I appreciate his sentiment and used to agree with his argument in principle, but now that I’ve thought about the fact that only “special URLs” are OpenIDs, I think it’s worthwhile to give that class of URLs a specific name.

Consistency

Furthermore, one of the greatest threats to the viability of OpenID is an inconsistent user experience. Unfortunately, this manifests itself both when signing in to a malfunctioning relying party, or attempting OpenID authentication using an OP that an RP doesn’t support (e.g. Microsoft Health Vault currently supports three OPs).

Another manifestation of this problem is that OPs are not required to consume OpenIDs. Though there’s validity in this complaint, change should not be forced at the technical level, because it really should be up to each individual provider to determine whose credentials it’s willing to accept. Now that the majors (save Facebook) have all gotten into the OP game (most recently Microsoft), it really just seems a matter of politics and inertia that none have moved to accept the OpenIDs of their competitors in any significant way (that is, neither Yahoo, Google, or Microsoft allow authenticating against their respective services using one of the other’s OpenIDs — and no, Blogger doesn’t count and Google hasn’t really released their OP yet).

While I’m sympathetic to Allen Tom’s argument that more OPs is frankly better for the web, I’m not convinced that a Visa card is all that useful if none of the major department stores will accept it.

I certainly respect large providers’ desires to both minimize the potential for abuse and to wade through the legal morass around identity technologies, but I can’t see how becoming an OpenID relying party is any worse than letting people create accounts with arbitrary (and untrusted) email addresses.

Hopefully through both political pressure and success-in-the-wild over time, we will see the majors become relying parties to their competitors’ OpenIDs for accessing accounts, and over a longer period of time, enable the use of personal/private OpenID providers or delegated OpenIDs (e.g. factoryjoe.com).

Should we see this situation change, I think it’ll bring about a watershed migration to patterns established by the majors — leading to consistency in the OpenID sign up and sign in experiences, and consistency in what people expect of OpenID account federation, leading to increased credibility and use of OpenID generally.

Leadership

But let’s get real: all these issues are going to require, above all else, solid foresight and leadership and a commitment to pushing through the thorny political issues that can often scuttle the best intentioned technologies (consider HD-DVD and Blu-Ray).

For reasons beyond my grasp, the OpenID Foundation has not met up to my expectations of leadership. Despite considerable progress in some areas, large swathes of stagnation have come to subsume many of the organization’s initiatives. International progress, as overseen by the OIDF, is lacking, except where local chapters (such as in Japan and in some European cities) have taken matters into their own hands. Code improvements to the OpenID libraries has languished and implementation of OpenID in various platforms and open source projects seems non-existent. Marketing simply isn’t happening and even if it were, I’m not convinced that there’s consensus on what we should market. And only now, after research from Yahoo and Google confirm what many critics have said for a long time is there finally work being done to address OpenID’s usability pitfalls.

Now, I realize that technologists don’t always make the best politicians (or designers or marketers for that matter) but that we haven’t seen the kind of OpenID visibility, credibility, innovation and adoption in North America that has been seen in Japan suggests to me that we’re either on the wrong course, or no apparent course at all. Worse, I fear that certain companies are already dividing up the proverbial “identity pie” before the damn thing’s even been put into the oven — a situation that needs to be addressed immediately by prioritizing a series of steps that the OIDF will take to establish OpenID in the marketplace, set firm how it will support individuals and companies alike, plot out its administrative and advocacy agenda for 2009, make clear its budgetary outlook, and list the marketing, design, education and research initiatives it plans for the coming year.

Without a clear path forward, I think that a lot of otherwise positive energy will devolve into useless sniping and infighting. Without strong leadership, we risk marginalizing many of the gains we’ve made to date in establishing OpenID as a core building block of the open social web.

For comparison, consider the progress that has been made with OpenSocial: only a year ago, people dismissed it as a “Gadgets API” (which, arguably it was). Since then, a large coalition of the willing has gathered to support and develop the protocol (which is still far from perfect, but demonstrates steady progress towards a goal), even convincing that old salt David Recordon that what they’re doing is decent. When OpenSocial 1.0 is released (they’re at 0.8.1 right now), there will be a distributed social graph with over 350 million potential customers available to developers (compared with around 100 million on Facebook). While David is right to point out, with Microsoft coming on board, there’ll be well beyond half a billion OpenIDs in the wild, that doesn’t mean that our work is finished. Rather, it’s just begun, and David sums up our situation fairly well:

While this is great news from Microsoft, real web-scale adoption of technologies always faces a chicken-and-egg problem between developers and vendors. Developers don’t want to adopt a technology without buy-in from platform providers and platform providers don’t want to support a technology if developers won’t use it. We’ve largely been able to successfully avoid this concern with OpenID as it grew from roots in an open source community with lots of people and companies involved in making OpenID what it is today. There are now well beyond half a billion OpenIDs available on the web which means we can mark the first phase of OpenID adoption, platform support, as a success.

The next phase of developer adoption will not be measured in the number of OpenIDs or sites that support it, but rather user experience, accessibility, and seamlessness of integration into a wide variety of applications and experiences.

To that end, there will be an Internet Identity Workshop in Mountain View November 11-12 where many of the primary participants in the ongoing identity conversations will converge. Historically the event has been one of the most productive in the space and with all the recent OpenID news lately, I’m hopeful that many of the issues I’ve mentioned above will be addressed and progress will continue to be made.

I will continue to be a staunch advocate of OpenID and think that it’s best times are still to come, but not without a redoubling of focused effort around concrete and ambitious goals.

Politics is something that I normally don’t cover on my blog, but not for any particularly reason. I typically get more [publicly] worked up about technology and the economics and politics of technological development than I do about directly human-facing issues, but that’s not because I’ve ever lost sight of the fact that ultimately all this technology is intended to serve people, or that there are more important, and more visceral, issues that could be tackled for greater, or longer lasting effect. It’s just that I haven’t really felt like I had an articulate contribution to make.

Perhaps until now.

If you’re not interested in political discourse, that’s of course your prerogative and you certainly can skip this post. Personally, however, I’ve become increasingly interested in what’s going on in this country (my country), and increasingly enamored of political dialogue (however bereft of content as it sometimes is) as well as our representative democracy — an imperfect system to be sure, but one that at least, by and large, affords its constituents a voice in matters local, state and federal. And personal.

Here in California, we have a cagey system of democracy where voters are provided the opportunity to consider multiple arguments for and against several propositions presented on a ballot to determine numerous policies at both the state and local level. I voted absentee yesterday (as I’ll be traveling to Oceania later this week) and along with the ballot for the presidential election, there were two accompanying ballots, one for the state and one for the city of San Francisco, where I am a resident.

On the state ballot is Proposition 8, effectively an amendment to the California state constitution that would ban gay marriage by defining it strictly as a union of a heterosexual couple: one man, one woman.

I voted against this proposition. And I’ll tell you why.

Back in the day…

When I was a senior in high school (in conservative “Live Free or Die” New Hampshire), I supported an initiative to create a gay-straight student alliance, or GSA. At the time, I was on the staff of the newspaper and was more informed of the various controversies affecting my classmates, but I’ll admit, I was also pretty ignorant of other “lifestyles”. Still, if my parents taught me anything, tolerance and self-respect were a few of the more subtle lessons that must have stuck, which led me to support the effort.

As I had done for many of the school’s student clubs, I created a homepage with information on the GSA initiative and hosted it on my own website. I had also single-handed built my high school’s website (even though I couldn’t get any educator besides the dorky librarian to care) and inserted a banner ad into the site’s rotating pool of four or five ads promoting the other school club sites that I’d designed.

The ad for the GSA, which didn’t say much more than “Find out more” with a link off-site, was in rotation for several weeks when I was called down to the principal’s office to explain why I was announcing school policy without authorization. So it goes in the petri-dish of adolescent high school politics and unbalanced power relationships.

Rather than use this as an educational opportunity, the principal, who later became mayor of the city, decided instead to use this situation as a reeducational opportunity and externally suspended me for six days, meaning I wouldn’t be able to graduate.

I’ll cut to the chase in a moment, but in response, I took down the GSA ad — as well as the entire high school’s site (I was hosting that on my own server too — back in 1999 schools didn’t know what a “web server” was). I vowed that I wouldn’t turn over the site files until they’d written up rules governing what students were and weren’t allowed to post to the school’s site; meanwhile my mom threatened to sue the school.

My infraction was small beans (and eventually overturned) compared with the lawsuit that GLAD and the ACLU filed against the school district barring discrimination against school clubs. By the time the lawsuit was decided in favor of the students, I had graduated and moved off to Pittsburgh, but the experience, and impression that it left on me, has resonated since.

…history repeating

None of these contested issues really consume you until you’re personally affected, as I was in high school, and today I feel equally affected by this proposition, but more capable of doing somethi