Finally, a pymsn release

January 12, 2008

Hello everyone,

The Pymsn developers are happy to introduce you to Pymsn 0.3, the brand new version of the python MSN protocol free implementation. This release is the result of about one year of unsteady development, during which Pymsn was completely rewritten to support the latest version of the protocol: MSNP15.

We are proud to announce that we reached our goal, and you can grab the tarballs from here: http://telepathy.freedesktop.org/releases/pymsn/

Features brought to you by this release are :

- Best MSNP2P implementation : avatars and custom emoticons
- Yahoo messenger contacts support : chat and presence
- Live Address Book support : full contact and group management
- Server side contacts aliasing
- Offline messages support
- MSN Spaces
- Content Roaming support : avatar, display name and personal message storage
- Full support for HTTP and HTTPS proxies
- Support HTTP Polling transport
- No threads, fully asynchronous
- Caching system

We would like to thank everyone who helped to make this release happen.

The new telepathy-butterfly is out as well : http://telepathy.freedesktop.org/releases/telepathy-butterfly/


the Erlang thing went well

December 5, 2007

Yesterday, four of my classmates and myself gave an hour long lecture on Erlang/OTP. That lecture was followed by a two hours long lab. It’s been a while I wanted to share about Erlang, I read a lot about it but unfortunately, I never found some project I was interrested enough to dive in. I now have an idea but the usual problem is showing up : not enough time.

So the presentation covered the Erlang language, with its roots (the Ericsson lab, industrial use), its paradigms (functionnal and concurrency oriented) and syntax (that reminded Prolog and OCaml to the attendance). I mentionned the capabilities offered by the OTP framework (generic behaviours, supersivion trees, hot code swapping, …) linking all that to reliability and availability concepts as we got a fault tolerance lecture few weeks ago.

The lab was based on a simple messenger application from the Erlang getting started tutorial. Given the amount of time, I structured it as a “fill in the blanks” exercise. All went OK and people seemed happy with their working system, at the end. That was really a great feeling to explain processus interaction and deal with students mistakes while helping them. It’s also great to open their eyes on an unknown technology, as they now could choose to pick it up to solve some industrial challenges they could run into during their career.

On my side, I’m now gonna find time to start that Erlang project I think about. I really need to get skills on it as it’s definitively part of my fields of interrest. That could help to find a related job later.


pymsn, butterfly and empathy

September 17, 2007

This weekend, Ali got his msnp2p stack polished so we could work on some p2p integration in pymsn. This led display pictures (aka avatars) to land into the library. That means we are really close to release the new version of pymsn on which we’re working for several months now.

Today, as we were implementing display pictures in pymsn, I wrote the telepathy-butterfly (the MSN connection manager for Telepathy based on pymsn) part for avatars, which was a good opportunity to test the new feature. This closes the list of features to implement into butterfly before a release as well, since I got groups and aliasing (you can nickname your buddies) working in the past days. Contact handles format was changed to support Yahoo contacts in butterfly (as pymsn, covering MSNP15, allows it).I’ve tried to integrate the pymsn offline messages feature into butterfly but I wasn’t pleased with it so it’s sleeping in a separate branch for now. However, I’d like to get full membership groups management too to be able to add/remove contacts, block them, etc.

The next screenshot shows empathy displaying my personal MSN account using telepathy-butterfly. I’m really glad that we are finally getting concrete things to show :)

screenshot-contact-list-1.png


an OpenIM proposal

September 4, 2007

I finally wrote a freedesktop.org project request for the OpenIM thing that Ali and I (and certainly lots of other people involved in IM protocols reverse engineering) would like to see come up. The request is a bug report on freedesktop bugzilla, please bring your observations and ideas! Here is quoted the text of the proposal :

This is about a new project that I would like to see come up under the fd.o banner.

I’m part of the pymsn project. Our goal is to provide a full implementation of the latest MSN Messenger protocol for interoperability purpose. So far the project is kind of hosted by Telepathy since it’s the base library for telepathy-butterfly but this is clearly not the place where it should be.

Based on that observation, we would like to build an OpenIM initiative. Such a project would aim to the gathering of people working on the opening of closed instant messaging protocols.

One of the most important goal in this would to have centralized shared documentation on protocols and a clear follow up of features implemented in each project.


Activity around the pymsn project

August 29, 2007

I begin to hear about some projects using our beloved python MSN Messenger library : pymsn. As you could have guessed, most of them are clients.  There is CupsAndString, a command line client written in 128 lines of python by Alsuren, LiMSN which has a wxWidgets interface and finally Open Live Messenger, a GTK+ client written by Julien Enche (Trapamoosch). Thanks to all of the client developers for your feedbacks and for providing examples of how to use the library ;) .

Of course, pymsn is also the base library for the telepathy-butterfly Telepathy connection manager. There is a development version of butterfly but most of the pymsn new/refreshed features will be integrated in it just before the release because it’s not a weighty work to do and all development efforts should better go for pymsn.

If any reader of this post is interested in learning more on the pymsn project, feel free to join #pymsn on freenode to have a good chat. Contributors of all sorts (developers, testers, documentation writers, …) are welcome too :)


pymsn article : the crack-powered address book

August 24, 2007

This article attempts to provide an overview of the contact management system shipped with the pymsn library. In the following I assume you are already familiar with the basic pymsn API that allows you to connect to the server and attach the various event handlers.

Contact management is handled by the AddressBook object which manipulates AddressBookStorage, PendingContact, Group and Contact objects. I will try to describe each of these objects API and related event interfaces.

When you develop a client using pymsn, you generally start by instanciatiating the Client class, and then instanciating the event handlers you created. The Client instance holds an instance of the AddressBook that can be accessed through the address_book property:

client.address_book

The AddressBook instance provides a set of methods allowing you to interact with you address book stored remotely on the MSN servers, in fact each method call result in one or more SOAP requests (ouch). The results of those actions are then reflected on pymsn internal representation of data and the related methods from the AddressBookEventInterface are called to notify you that the changes have been done (contact added, group deleted, …). So all you have to do is to inherit from that event interface and fill the methods with your own needs then instanciate your instance.

Now let’s have a closer look on all those entities…

AddressBook

The AddressBook is the central object in contact management. It owns four important properties :

  • contacts is an AddressBookStorage instance. It’s a collection containing Contact objects (actually representing the address book contact list). This object is very useful to request a subset of specific contacts using chained method calls.
  • state is an AddressBookState value which describes the synchronization state of the address book. The address book is said to be synchronized when the local AddressBook data reflects excactly what’s stored on the server. When using the Client class, the given AddressBook instance is automatically synchronized at startup (else you would have to call the sync method).
  • groups is a dictionary of Group objects indexed by their ids. It contains all the groups that are defined for that address book.
  • pending_contacts is a set of PendingContact. Those objects are simple ones containing information on the contacts that asked to be added to the address book. Those pending invitations are to be accepted or declined.

API description

The API is short, simple, and method actions are self explained by their name and the name of their parameters, however when can here explain some bloody details.

Invitations
accept_contact_invitation(self, pending_contact, add_to_contact_list=True)

Method used to accept a pending invitation. It takes a PendingContact object from the pending_contacts set. You can specify if you want to add the new contact to your contact list (default behavior) or not. The pending contact will then be automatically removed from pending_contacts.

decline_contact_invitation(self, pending_contact)

Method used to decline a pending invitation. It takes a PendingContact object from the pending_contacts set. The contact will be blocked and won’t appear in you contact list. The pending contact will then be automatically removed from pending_contacts.

Contacts
add_messenger_contact(self, account, invite_display_name='', invite_message='')

Method used to add a “messenger contact” to the contact list. The “messenger contact” means a MSN or Yahoo contact (as opposed to “mail” or “mobile” contacts which are pending soon-to-be-added pymsn features). The main argument given to this method is the account (a valid email address). Giving a Yahoo domain email address, pymsn will try to add it both as a MSN account and a Yahoo account. You can also specify a display name and a message that may be displayed by the client used by the contact receiving the invitation.

delete_contact(self, contact)

Method used to delete a contact from the contact list. The given argument must be a Contact object from the contacts collection. The contact will then be automatically removed from contacts.

block_contact(self, contact)

Method used to block a contact. The given argument must be a Contact object from the contacts collection.

unblock_contact(self, contact)

Method used to unblock a contact. The given argument must be a Contact object from the contacts collection.

Groups
add_group(self, group_name)

Method used to add a group to the address book.

delete_group(self, group)

Method used to delete a group from the address book. The given argument must be a Group object from the groups dictionary.

rename_group(self, group, new_name)

Method used to rename a group. The first argument must be a Group object from the groups dictionary. The second one is a string.

add_contact_to_group(self, group, contact)

Method used to add a contact to a group. The first argument must be a Group object from the groups dictionary. The second one must be a Contact object from the contacts collection.

delete_contact_from_group(self, group, contact)

Method used to delete a contact from a group. The first argument must be a Group object from the groups dictionary. The second one must be a Contact object from the contacts collection.

AddressBookStorage

An AddressBookStorage is a set that contains Contact objects. This storage object provides several way to request for contacts. Queries are method calls that can be separated in two families : method beginning with “search_by” return sub address books (which are AddressBookStorage too) based on a field selection and method beginning with “group_by” group contacts together in a dictionary, based on a field value. The class provides some methods to query on not trivial Contact attributes :

search_by_memberships(self, memberships)

This method returns an AddressBookStorage object that contains the Contact objects which match the given memberships parameter. That parameter is a bitwise OR of Membership lists. There are lists for blocked contacts, allowed contacts, contacts that got you in their contact list, etc.

search_by_groups(self, *groups)

This method returns an AddressBookStorage object that contains the Contact objects which are part of the given groups. Each group parameter must be a Group instance.

group_by_group(self)

This method returns a dictionary in which each key is a Group and its linked data an AddressBookStorage object containing the Contact objects which are part of the key group.

In addition, pymsn makes use of the python __getattr__ method to provide custom query methods. Those are based on the next two methods that perform queries on the Contact object attributes :

search_by(self, field, value)

This method returns an AddressBookStorage containing the Contact objects for which the field attribute matched the given value.

group_by(self, field)

This method returns a dictionary which associates values of field to the related Contact objects.

Since the “search_by” methods return AddressBookStorage objects, you can execute complex request chaining method calls. For instance, the next instruction will select all the Yahoo Messenger contacts from the address book and then group them in a dictionary using their presence as keys :

address_book.contacts.search_by_network_id(pymsn.profile.NetworkID.EXTERNAL).group_by_presence()

Contact, PendingContact and Group

Those objects are the ones manipulated by the address book but the best thing to do is to refer to the API documentation to learn more about them so they won’t be described here.

AddressBookEventInterface

As with the other pymsn event interfaces, you’ll have to implement that interface and attach it to the Client. We won’t describe each method of the interface : almost each of the AddressBook methods fires an event method when its task is accomplished (the given arguments are then the concerned Contact or Group objects). We can however note that an event method exist to acknowledge you when there’s a new pending contact and that the errors that could occur performing the task (for instance when trying to add a contact who is already in the address book) are redirected to the client error handler.

Conclusion

Pymsn provides a powerful contact management through the address book. The API is still young and could certainly be improved : feel free to submit patched or bug reports. Feature requests are welcome too, we already have some pending ideas to be implemented : contact informations update (concerning mail addresses, birthdate and that kind of stuff), possibility to automatically add a contact to groups when adding him to the address book, etc.

If anybody find it useful, that blog post will become a page somewhere and will be updated to each API change.


Status of pymsn services development

August 15, 2007

Here we are :

  • Implementation of the address book is finished : the integration in msnp and the rest of pymsn happened last weekend. We are now massively debugging the behaviors of the service thanks to the help of Julien Enche (Trapamoosch) who is developing a client, daily integrating each of our feature adds, thus detecting lots of bugs. Thanks again Julien for your assiduity to post bug reports.
  • Implementation of the content roaming service is finished : this is fresh from tonight but all my tests went good. We are now able to retrieve the display name, personal message and display picture stored on the server as well as we can publish those. We now need to find a way to properly integrate this within the client stuff.
  • Implementation of the offline messages service is close to be finished : we’re able to get the message headers and fetch their content. The method used to send messages is the only thing missing for now but that’s just a matter of time since there’s no difficulty around. Little work on integration with the rest of the library is still to be done.

Future : it seems that concerning services, we’re closer to the end than ever. Anyway, there’s a huge thing missing for now : Spaces. That service is the last existing. It allows to user to interact with all the blogging stuff in the MSN/Live network. Supporting spaces would be a great plus for clients. So we’re gonna start to work on this. Nothing is done yet and there’s a lot of reverse engineering effort needed. [Update : more on this in Youness's comment]

Now a screenshot taken from the excellent Baobab, a disk usage analyzer, concerning the library directory from the rewrite :

Pymsn disk usage analysis

We easily figure out that the service package represents about 56% of the whole library. This is due to the huge amount of lines of code we have to write to set a new service up. We can be glad with our SOAP stack since we now can generate anything we want in a clean enough way but services aren’t a fun part to work on : once the reverse engineering is done, you just have to follow the rules, fit the design, and you finally get your stuff working. I hope I’ll be able to contribute more on other parts of the library soon.


the iSight of the moon

August 3, 2007

I finally got my built-in webcam working on the crackbook. Ok, that was just the matter of looking for an HOWTO on the Ubuntu forum and that just worked in 3 minutes. I’m now at my summer job (in a food-for-farm-animals factory) but they kinda gave nothing to do to me so I brought the laptop, plugged the network in, stole the proxy credentials and now I can work on my stuff (being paid). I hope that situation will end soon and that they’ll find me something to work on because I have 4 weeks left here. But, for now.. let’s play with some multimedia stuff :D

isight_capture


pymsn scenarios

July 26, 2007

Not a scoop, the MSN address book works using SOAP services. Those services provide methods to manipulate data on the server. However, when you want to do something like block/unblock a contact or synchronize the lists, you have to call several different methods as a sequence. This can be an issue in pymsn since the SOAP calls are asynchronous. Trying to understand what the “partner scenarios” mean, I finally find a solution out.

In the MSN SOAP calls, there’s an header field name “partner scenario”. That field seems to have a different content related to the moment it’s used (when adding a contact, when accepting a contact request…). Even if the server doesn’t seem to discriminate on that field, the new service stack allow us to use the right partner scenario at the right place. For me, partner scenarios are all about the context : it’s not that bad to know in what context a method is used.

So my idea was to create scenarios objects in pymsn. A scenario object handles an action like “block a contact”, to and fro, calling several SOAP methods if needed. Since the calls are asynchronous and we’re using callbacks to know when the response is available, a scenario object defines kind of a “chain of callbacks” that finally ends up calling the scenario callback. A scenario object can be executed, actually it’s very close to the command design pattern.

The high level user won’t manipulate the scenarios : scenarios help us to have a clean address book object which just has to create scenario objects, fill them with the needed data, and execute them.


the new pymsn service stack

July 26, 2007

Bored of trying to generate the ugly SOAP stuff from the MSN protocol with SOAP python libraries (SOAPpy, ZSI or our very own internal library), we finally came up with a new way to communicate with web services. We are now using massive string formatting! Each SOAP method is described in a ‘description’ file which contains its header and body. On top of that, classes inheriting from a SOAPService class are able to generate the SOAP method calls and the response or error callbacks handle. There is one of these class for each real service. At the top level, the user will for instance only have to care about an AddressBook object that uses the address book (contact management) and sharing (membership management) services and where all the data is normalized. Tough the “massive string formatting” way could sound a bit tricky, we managed to get a clean and flexible enough stack which allows us to do whatever we want without creating a mess. It was sweet to rely on it to build the new address book support to pymsn during GUADEC (more on that later).

the new pymsn service stack


Follow

Get every new post delivered to your Inbox.