HTTPS Login with Cookies

August 25, 2010

I was trying to write a python script which would login to a remote web page, and then request some data. The login was a standard HTML form, so I had to POST to a url with the appropriate form fields.

Using python httplib, everytime I did the POST for the login, I was getting the login page HTML returned again, with a 200 status. To solve this I switched to the httplib2 library - I couldn't get this to work until I switched to this.

Once I had login working, I had trouble requesting the data url. I believe this was to do with the .NET server I was communicating with using cookies to enforce the login session. As I didn't pass the cookie into my data request, it didn't think I was authenticated.

Here's the code that I came up with:

import httplib2, urllib


lLoginHeaders = {'Content-type': 'application/x-www-form-urlencoded'}

http = httplib2.Http()
lResponse, lContent = http.request(lLoginUrl, 'POST', headers=lLoginHeaders, body=urllib.urlencode(lLoginParams))
print lResponse.status

lDataHeaders = {}
lDataHeaders['Cookie'] = lResponse['set-cookie']
lResponse, lContent = http.request(lDataUrl, 'GET', headers=lDataHeaders)
print lResponse.status

print lContent

This code first of all does a POST to a login url. It then gets the cookies from the login response, and passes them into the GET for the data.