Proposal for Authenticating Web Callbacks

Web callbacks are an unapologetically cool idea, and for me they are a welcome advance beyond polling in terms of both concept and efficiency.

In polling designs, a client with no knowledge of the service's state is responsible for detecting when the service changes.

This means that the client will repeatedly ping the service to check if the service's state has changed, and it has no way to determine beforehand if it is wasting its (and the service's) bandwidth or if there is new data to act on.

Polling is akin to a busy wait.

On the other hand, Web callbacks are very much like the callbacks approach to programming, seen most frequently in JavaScript as a work-around for single threaded implementations^1.

Faced with this fairly novel idea, a question came to mind: how should web-callback services authenticate client requests? In a world without authentication, a service could be used as a vector for DOS attacks, and generally it is quite rude to start sending messages to a client without verifying that it is indeed the client itself who has sent the request.

After thinking about this, I have two proposals that I believe are both viable:

1. A three-way handshake which would begin with the client requesting a service, then the service would send the specified client's hostname a shared secret, which the client would then use to sign requests.

2. Require an explicit allow line in robots.txt for the given API. Thus site owners would add something along this line in their robots.txt:

   Allow: /api/callback/
   

   and then the service would verify such a line exists for the given callback before adding the site to its list of verified callbacks.

   (This could potentially be done in a new convention file callbacks.txt instead.)

I tend to prefer the second option (and, in particular, would prefer having a separate callbacks.txt), as it doesn't require any additional development work on consumers, and wouldn't be particularly difficult to implement.

Thoughts?