Skip to content
Apr 8 / Erik Wilde

Feeds as Query Result Serializations

, ,

Many Web-based data sources and services are available as feeds, a model that provides consumers with a loosely coupled way of interacting with providers. The current feed model is limited in its capabilities, however. Though it is simple to implement and scales well, it cannot be transferred to a wider range of application scenarios. This paper conceptualizes feeds as a way to serialize query results, describes the current hardcoded query semantics of such a perspective, and surveys the ways in which extensions of this hardcoded model have been proposed or implemented. Our generalized view of feeds as query result serializations has implications for the applicability of feeds as a generic Web service for any collection that is providing access to individual information items. As one interesting and compelling class of applications, we describe a simple way in which a query-based approach to feeds can be used to support location-based services.

250px-Line3174_-_Shipping_Containers_at_the_terminal_at_Port_Elizabeth,_New_Jersey_-_NOAAFeeds as Query Result Serializations is a technical report published today. it describes what we think will be the next step in the evolution of lightweight web services. in the same way as containerization has revolutionized the global trade of physical goods, feeds are already essential as a lightweight and universal way of sharing machine-readable information, and this trend will continue.

one of the major missing pieces in the atom landscape today is the lack of well-defined query features, and we argue that such a capability would make atom feeds much more flexible in a wide variety of scenarios. this can be as simple as querying feeds by category (something already possible with GData), or as sophisticated as spatial queries when considering geo-referenced collections. more specifically, the step from the Internet of Things to the Web of Things, i.e. making the step from enabling connectivity to a web architecture for networked things, probably will be made possible by smart ways of extending and combining feed technologies. after all, many of the data streams produced by sensors or other pervasive technologies can be readily mapped to feeds, and exposing stream processing capabilities as queryable feeds would bridge the gap between web-based information streams, and information streams originating in the physical world.

Filed under Research, Technology
  • vlad

    Thanks for the post Erik. Indeed, I’ve been thinking a lot about these last couple of days, but what I feel is missing is a push mechanism in a publish/subscribe context. I’ll be reading your paper you mentioned, and I think from a user perspective, searching and finding particular data and feeds is a must.

    I’ve been working recently on streaming and eventing in the context of sensor networks, and what is missing to build a truly scalable and multi-users sensor net is a mechanism where different user can subscribe independently to event from all kinds of device. but requires moving away from the client server model, to a every thing is both a client and a server. This is the area I’ll be looking more in detail, because I think having an atome-based xmpp would be really useful for most applications for monitoring and data collection.

    Any thought on this push/pull “dilemma”?

  • http://dret.net/netdret/ dret

    http://www.infoq.com/interviews/tim-bray-future-of-web and forward to 04:20; there are many cases where push is really not a good idea. not if you want massive scalability outside of your little event-oriented bubble.

  • http://www.vladounet.com Vlad Trifa

    I’ve been reading your paper this morning and understand exactly your point. Of course if you have 2-3 users that’s very fine. On the other hand what I think is more problematic is more about the timely information, like I don’t want the police client to refresh the feed of my home intrustion detector everz couple of hours, I want to know that right away, within seconds.

    I think an awesome solution for that is for devices to push their data once and AT once to an intermiediate server or gateway. And the gateway (or server farm or whatever that is not resource-limited like sensor nodes) would take care of helping the scalability of the system.

    I see two use cases, and feeds for me lack the eventing capability in their current form (or I see the sensor pushing the entry with atompub into my client, then it works, but we’re less loosely coupled). I think http intermediates could very well help to find something in between. I’d really like to investigate these tradeoffs with more data and sensors and actual uses cases, to provide some more quantitative data on this issue.

  • http://dret.net/netdret/ dret

    i think exploring a scenario with AtomPub-based push would be very interesting, in particular when looking at intermediaries that might be pushed to and then push push out to a different set of clients. consider the iPhone AppStore: update notifications are pushed to clients, but it actually would not be necessary for app developers to really push their update to apple. or maybe they do, but that part should be treated independently from the notification architecture for the clients.

    so yes, looking into push makes sense in some scenarios. and it does decrease loose coupling, but that’s the price you have to be willing to pay for the immediate notification. but i think it really is important to keep in mind that push-based PubSub-scenarios are considerably more expensive to maintain and operate than pure pull (and they scale poorly), so one part of such an architecture should be to make sure that servers as well as clients can easily switch between push and pull styles, or maybe to only use push for special (paying or otherwise essential) subscribers, while others can get the same events, but they have to pull.
    architecturally speaking, this is not trivial, but if it’s well done, it could be a core part of the infrastructure of the web of things; so i am looking forward to discuss this in excruciating detail ;-)