***** UPDATE: The script in this article no longer works due to Google authentication changes. Please use the new script described here: Motion Google Drive Uploader for OAuth 2.0
I'm using the brilliant Motion software with my Raspberry Pi to capture goings-on around my house. This blog post: Battery powered, Wireless, Motion detecting Raspberry Pi and this blog post: Raspberry Pi Webcam explain how.
I set up Lighttpd to point at the directory where Motion was creating video files so that I could go and check periodically to see if any new events had been captured. This works well - I just set up my home router so I could access the http over the Internet and then download the AVIs to my laptop or whatever. The down side to this is that you have to pro-actively look to see if there are new videos and secondly it puts additional strain on the Raspberry Pi to host the files for repeated download.
So, I have created a Python script to upload the AVI to Google Drive, get a URL to view the video on line and then email this using the same GMail account to an address. To keep the Raspberry Pi tidy the script then deletes the local file.
If you have the Google Drive Sync application running too you can see the files as they arrive which is fun:
Source
#!/usr/bin/python2 ''' Created on 6 Jun 2012 @author: Jeremy Blythe Motion Uploader - uploads videos to Google Drive Read the blog entry at http://jeremyblythe.blogspot.com for more information ''' import smtplib from datetime import datetime import os.path import sys import gdata.data import gdata.docs.data import gdata.docs.client import ConfigParser class MotionUploader: def __init__(self, config_file_path): # Load config config = ConfigParser.ConfigParser() config.read(config_file_path) # GMail account credentials self.username = config.get('gmail', 'user') self.password = config.get('gmail', 'password') self.from_name = config.get('gmail', 'name') self.sender = config.get('gmail', 'sender') # Recipient email address (could be same as from_addr) self.recipient = config.get('gmail', 'recipient') # Subject line for email self.subject = config.get('gmail', 'subject') # First line of email message self.message = config.get('gmail', 'message') # Folder (or collection) in Docs where you want the videos to go self.folder = config.get('docs', 'folder') # Options self.delete_after_upload = config.getboolean('options', 'delete-after-upload') self.send_email = config.getboolean('options', 'send-email') self._create_gdata_client() def _create_gdata_client(self): """Create a Documents List Client.""" self.client = gdata.docs.client.DocsClient(source='motion_uploader') self.client.http_client.debug = False self.client.client_login(self.username, self.password, service=self.client.auth_service, source=self.client.source) def _get_folder_resource(self): """Find and return the resource whose title matches the given folder.""" col = None for resource in self.client.GetAllResources(uri='/feeds/default/private/full/-/folder'): if resource.title.text == self.folder: col = resource break return col def _send_email(self,msg): '''Send an email using the GMail account.''' senddate=datetime.strftime(datetime.now(), '%Y-%m-%d') m="Date: %s\r\nFrom: %s <%s>\r\nTo: %s\r\nSubject: %s\r\nX-Mailer: My-Mail\r\n\r\n" % (senddate, self.from_name, self.sender, self.recipient, self.subject) server = smtplib.SMTP('smtp.gmail.com:587') server.starttls() server.login(self.username, self.password) server.sendmail(self.sender, self.recipient, m+msg) server.quit() def _upload(self, video_file_path, folder_resource): '''Upload the video and return the doc''' doc = gdata.docs.data.Resource(type='video', title=os.path.basename(video_file_path)) media = gdata.data.MediaSource() media.SetFileHandle(video_file_path, 'video/avi') doc = self.client.CreateResource(doc, media=media, collection=folder_resource) return doc def upload_video(self, video_file_path): """Upload a video to the specified folder. Then optionally send an email and optionally delete the local file.""" folder_resource = self._get_folder_resource() if not folder_resource: raise Exception('Could not find the %s folder' % self.folder) doc = self._upload(video_file_path, folder_resource) if self.send_email: video_link = None for link in doc.link: if 'video.google.com' in link.href: video_link = link.href break # Send an email with the link if found msg = self.message if video_link: msg += '\n\n' + video_link self._send_email(msg) if self.delete_after_upload: os.remove(video_file_path) if __name__ == '__main__': try: if len(sys.argv) < 3: exit('Motion Uploader - uploads videos to Google Drive\n by Jeremy Blythe (http://jeremyblythe.blogspot.com)\n\n Usage: uploader.py {config-file-path} {video-file-path}') cfg_path = sys.argv[1] vid_path = sys.argv[2] if not os.path.exists(cfg_path): exit('Config file does not exist [%s]' % cfg_path) if not os.path.exists(vid_path): exit('Video file does not exist [%s]' % vid_path) MotionUploader(cfg_path).upload_video(vid_path) except gdata.client.BadAuthentication: exit('Invalid user credentials given.') except gdata.client.Error: exit('Login Error') except Exception as e: exit('Error: [%s]' % e)
Installation
If you haven't already got Python and Pip installed then do it now. On my Arch Linux Raspberry Pi I did this:
Next get the Google gdata library:
At this point it's worth testing the script e.g.
Enjoy!
pacman -S python2 pacman -S python2-pipNote: This script has been tested on Python 2.7.3
Next get the Google gdata library:
pip2 install gdataNow download the two files above to somewhere on your machine. Make uploader.py executable:
chmod +x uploader.pyOpen uploader.cfg to enter your settings:
[gmail] # GMail account credentials name = My Name user = gmailusername password = gmailpassword sender = me@gmail.com # Recipient email address (could be same as from_addr) recipient = me@gmail.com # Subject line for email subject = Motion detected # First line of email message message = Video uploaded [docs] # Folder (or collection) in Docs where you want the videos to go folder = motion [options] # Delete the local video file after the upload delete-after-upload = true # Send an email after the upload send-email = trueMake sure you create a folder in your Google Drive to match the setting in the uploader.cfg file. This is the folder where the videos will go.
At this point it's worth testing the script e.g.
./uploader.py /etc/motion/uploader.cfg /srv/http/motion/cam1/85-20120608194940.aviIf all's well you can now change the Motion setting like so:
on_movie_end /root/py/uploader.py /etc/motion/uploader.cfg %f
Enjoy!
Amazing articles! I have been following your installation of the webcam and this has been a great reference for info for the Raspberry Pi!
ReplyDeletenice one! thanks a lot :)
ReplyDeleteI try to make use of your script on an ubuntu box, but after installing python-gdata, i have an error on importing gdata.data?
no module named data...
I have to fix that
Fantastic stuff and worked a treat, I love it.
ReplyDeleteMany thanks for the amount the work that's clearly gone into this and that you chose to share it.
Very much appreciated
Jeremy, many thanks for putting these Pi tutorials together!
ReplyDeleteI've got Motion up and running and am now trying to get the GDrive script to upload the video/images. When I run the script I get the generic "Login Error" and I'm not sure how to troubleshoot further.
I've done a bit of troubleshooting up to this point, including pointing the script to a non-existent folder on gdrive, which led to me getting the "Could not find the %s folder" response. To me, this rules out a log-in problem.
Any ideas?
@thehill06 looks like you might have to dig deeper into the gdata libraries to work out what's going on here - I wonder if there's more info behind that gdata.client.Error exception? In these situations I would pepper the code with print statements to see where it gets to before it falls over to hone in on the problem. Good Luck!
ReplyDeleteGot it working! This is brilliant. Thanks for your work and I look forward to seeing your other Pi projects.
ReplyDelete@thehill06 Great! Check out my new post that hooks up LEDs and buttons to Motion: Raspberry Pi GPIO and Motion
ReplyDelete@thehill06, I've got the same problem with "login error", however I'm not too sure where to start to debug the problem. What did you do to resolve the issue?
ReplyDelete@ Xen, it turns out when I was doing my test upload I was trying to upload an image rather than a video file. Once I pointed the script to a video the issue was resolved.
ReplyDeleteTanks for that excellent work! But i still have a litle problem...
ReplyDeleteWhen i execute the python script, i get an error:
Error: ['DocsClient' object has no attribute 'GetAllResources']
Can anyone help me?
Greetings from germany phate127
Does anyone have an example of how to upload an image instead of the avi to google.
ReplyDeleteI've managed to get it to upload a jpeg by changing the doc type to image/jpeg, but viewing in google docs just opens up a blank document.
Hi Jeremy, thanks for the post - using this for my homebrew burglar detection system! A quick question - did you install a desktop environment and do your editing from a text editor or did you just use nano to edit the python scripts from the terminal?
ReplyDelete@lewis I just use nano. I always connect in over ssh using PuTTY to do everything on the pi. It means less to install and more resources available. I also use svn to keep the code safe and then I can write it on my laptop and update on the pi.
ReplyDeleteThanks for the quick response Jeremy - that clarifies it for me! Great tutorials.
ReplyDeleteTanks for that excellent work! But i still have a litle problem...
ReplyDeleteWhen i execute the python script, i get an error:
Error: ['DocsClient' object has no attribute 'GetAllResources']
Can anyone help me?
Greetings from germany phate127
Try using the later version of gdata. I got this error with 2.0.14, but 2.0.17 solved it.
Very nice work Jeremy. I have completed most of this setup, except the video upload. When testing the script as you describe it works well. However when adding into motion.conf I see that we are not specifying the full path and video name to upload. Filename I can understand, but where does your script get the path to where motion stores the video files?
ReplyDeleteI see the variable "video_file_path" but dont see where it is defined.
Many Thanks.
Jeremy, please ignore my previous comment, have fixed it - IT WORKS !!
ReplyDeleteMany thanks for your efforts.
Hi Jeremy
ReplyDeleteJust got Error: ['DocsClient' object has no attribute 'GetAllResources']
ubuntu server 12.04 LTS
python 2.7.3-0ubuntu2
python-gdata 2.0.14-2
any solution??
thanks
@cvaldess - I haven't seen that error before but I suspect it is something to do with the gdata install. You could try doing some other basic gdata things from the google tutorials to see if it's working.
DeleteHi Jeremy,
ReplyDeleteThanks for the great post. I am got everything running but stuck on the last step. I can send force the upoloader to copy the video manually.
But it does not work when the motion is running. It only uploads the vipdeo when I stop the motion by pressing control+c
Any clues ?
Thanks
Mohsin
Also I only get the following trace when I press control+c after which it sends the file.
ReplyDeleteExecuting external command '/home/mohsin/Downloads/uploader.py /home/mohsin/Downloads/uploader.conf /tmp/motion/01-20120909040044.avi
And I dont see this kind of trace while the motion is running.
Thanks,
Mohsin
Ignore my earlier comments. It does take a bit of time to send the message when the motion is running. It actually doe send the message.
ReplyDeleteJeremy, run some examples from gdata-docs and work ok, think they don't support GetAllResources any more.. but not a python programer. thxn
ReplyDeleteHi,
ReplyDeleteThanks for the script/blog post(s) - all very useful.
Just wanted to highlight a minor-minor bug (sorry!) that might help other people...
self.client.client_login(self.sender, ...
should be
self.client.client_login(self.username, ...
That way it will log in with the username and not the sender e-mail address (in the case they are different) - that had me for a good while trying to figure out if Google's two-factor auth was to blame (I was using an app-specific password).
Many thanks,
Kenny
Hi, thank you for the great articles, I've successfully managed to get motion working on a fixed IP address and get to a point where i have the two scripts in directories on my RPi.
ReplyDeleteI'm however having trouble with running and testing the 'uploader.py' script. I keep getting an error saying '-bash ./uploader.py: No such file or directory'
I've only been playing with the Linux shell for a week and can't say I'm finding it as easy as I would have liked, so any help would be much appreciated!
Thanks
Great script, Thanks!
ReplyDeleteBut could it be made to upload JPEG images instead? I've changed the MIME and type="images", however it seems Google tries to convert the image on upload. The URL needs to have '?convert=false' added at the end when uploading, but I can't figure out how. Any help?
All working very well. I modified to upload jpegs instead of avi's:
ReplyDeletedoc = gdata.docs.data.Resource(type='image', title=os.path.basename(video_file_path))
media = gdata.data.MediaSource()
media.SetFileHandle(video_file_path, 'image/jpeg')
One question though, I then thought I'd add the URL to the mail but the links don't work. They begin
https://docs.google.com/feeds/default/contentmedia/folder
and when accessed I get Invalid Request URI.
Any ideas?
My jpegs are still coming in at 1kb, and as a .gdoc document. I can't figure out how to add the "?convert=false"
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteTo add the ?convert=false while still posting to a collection, make @southscanner's changes, then
ReplyDeletereplace
doc = self.client.CreateResource(doc, media=media, collection=folder_resource)
with
create_uri = folder_resource.get_resumable_create_media_link().href + '?convert=false'
doc = self.client.CreateResource(doc, create_uri=create_uri, media=media)
See
http://packages.python.org/gdata/docs/api.html
--and--
http://planzero.org/blog/2012/04/13/uploading_any_file_to_google_docs_with_python
Thanks for this script!
ReplyDeleteThe uploader works when running the script manually but not when the video has finished recording. I am using the exact same paths in both, except instead of ending with the filename in the config file I end with %f
Any ideas?
Thanks for the scrip.I have the same problem as my previous poster, please help.
ReplyDeleteVery good!! Thanks!
ReplyDeleteNot just for Pi lol. What a great post! Thanks Jeremy as this has helped me set up my motion setup with 4 webcams sending just the ones I need to Google Drive in photos. Great Post !
ReplyDeletefor wheezy:
ReplyDeletepython not python2
pip not pip2
change first line of script (interpreter) to python2.7
and works like a charm
I am using the ip camera to ftp the images to pi then upload it to drive. I can manually upload the images one by one but not when I use motion.cfg. I got both scripts and the jpgs are in one directory.
ReplyDeleteSK
Hi, I have never used python or linux before and was wondering if you could explain exactly where to plug in the filenames/paths. I have tried a millions combos and am very frustrated.
ReplyDeleteany help is greatly appreciated.