Weblog Tomasza Przechlewskiego

Wysyłanie posta na blogger.com z wykorzystaniem Google API

GoogleCL przestało działać, bo Google przestało obsługiwać wersję OAuth 1.0. Ponadto, wygląda na to, że dostosowanie tego użytecznego narzędzia do wersji OAuth 2.0 bynajmniej nie jest trywialne na co wskazują liczne (ale do tej pory bezskuteczne) prośby i wołania o aktualizację GoogleCL, które można znaleźć w Internecie.

Ponieważ poszukiwania w miarę podobnego zamiennika zakończyły się niepowodzeniem, nie pozostało nic innego zmajstrować coś samodzielnie. Autoryzację OAuth 2.0 mam już opanową -- obsługuje ją Pythonowy skrypt oauth2picasa.py. (Skrypt jest (zapożyczonym) fragmentem z projektu picasawebsync). Wystarczyło dorobić następujący prosty skrypt Perlowy (por. także: Publishing a blog post):

#!/usr/bin/perl
# *** Wyslanie posta na blogger.com ***
use strict;
use LWP::UserAgent;
use XML::LibXML;
use Getopt::Long;

my $profileID="default";
my $blogID = '1928418645181504144'; # Identyfikator bloga
my $blog_entry ;

## Na wypadek gdy ktoś ma kilka blogów moża podać na któr
## ma być wysłany post używając opcji -blog
GetOptions( "blog=s" => \$blogID, "post=s" => \$blog_entry) ;

if ( $blog_entry eq '' ) {
print STDERR "*** USAGE: $0 -b blog -p message (-b is optional) ***\n" }

## sprawdź czy post jest well formed:
my $parser = XML::LibXML->new();
eval {my $res_  = $parser->parse_string($blog_entry) };
if ($@) { die "*** Error parsing post message! \n"; }

my $ACCESS_TOKEN=`oauth2blogger.py`; # pobierz ACCESS_TOKEN
print STDERR "*** AccessToken: $ACCESS_TOKEN ***\n";

my $req = HTTP::Request->new(
  POST => "https://www.blogger.com/feeds/$blogID/posts/default");

$req->header( 'Content-Type' => 'application/atom+xml' );
$req->header( 'Authorization' => "Bearer $ACCESS_TOKEN" );
$req->header( 'GData-Version' => '2' );

$req->content($blog_entry);

my $ua = LWP::UserAgent->new;
my $res = $ua->request($req);

# Jeżeli coś jest nie tak poniższe drukuje verbatim:
# http://www.perlmonks.org/bare/?node_id=464442
# $ua->prepare_request($req); print($req->as_string); exit ;

if ($res->is_success) {
   my $decoded_response = $res->decoded_content;
   print STDERR "*** OK *** $decoded_response\n"; }
else { die $res->status_line; }

Wykorzystanie:

perl blogger_upload.pl -p 'treść-posta'

Treść posta musi być oczywiście w formacie xHTML i zawierać się wewnątrz elementu content, który z kolei jest wewnątrz elementu entry. Element entry zawiera także title określający tytuł posta, oraz elementy category zawierające tagi. Przykładem może być coś takiego:

<entry xmlns='http://www.w3.org/2005/Atom'>
 <title type='text'>Marriage!</title>
 <content type='xhtml'>
    <div xmlns="http://www.w3.org/1999/xhtml">
      <p>Mr. Darcy has proposed marriage to me!</p>
      <p>He is the last man on earth I would ever desire to marry.</p>
      <p>Whatever shall I do?</p>
    </div>
  </content>
  <category scheme="http://www.blogger.com/atom/ns#" term="marriage" />
  <category scheme="http://www.blogger.com/atom/ns#" term="Mr. Darcy" />
</entry>

Opisany skrypt jest tutaj: blogger_upload.pl.

url | Wed, 08/07/2015 17:48 | tagi: blogger, google, googlecl, oauth, oauth2, perl

Automatic SMS Alerts Using Google Calendar

The idea is to add an event to Google calendar with some short start time from now (say 15 minutes) and SMS reminder feature.

#!/bin/bash
# reminder via google with 15 minutes from now start time
#
MESSAGE="Neptune ALARM: something nasty has happened!"

# In case of service failure try 3 times:
for i in 1 2 3; do

# Compute 15min from now time with date:
NOW=`date "+%s"`
TIME_SHF1=$(($NOW + 900 ))
FWD_TIME1=`date +"%d/%m/%Y %H:%M" -d"@$TIME_SHF1"`; 

# Check for optional script argument 
if [ -n "$1" ] ; then MESSAGE="Neptune ALARM: $1" ; fi 

# use GoogleCL to add to calendar with 15m from now/1m reminder
google calendar add "$MESSAGE $FWD_TIME1" --reminder="1m"

RC=$?

if [ $RC -ne 0 ] ; then
   sleep 30;
else
   break ;
fi

done

The script uses GoogleCL (a python-based command-line utility for accessing Google). Installing GoogleCL is easy:

sudo apt-get  install googlecl
# or (in case there is no ready-to install package):
wget http://googlecl.googlecode.com/files/googlecl-0.9.13.tar.gz
tar -zxvf googlecl-0.9.13.tar.gz
cd googlecl-0.9.13
sudo python setup.py install

Note: googleCL is launched via google not googleCL command.

The first time one uses googleCL, ie:

google calendar add "test from neptune 22/03/2013 10:30:22"

it will prompt for one's Google username. One has to type it in and hit enter. Next the user is asked to grant access permissions.

On non-gui systems (text-based) w3m browser usually will be launched. When connected to google one has to press q y and the text silimlar to the following will be displayed:

Please log in and/or grant access via your browser 
at https://www.google.com/accounts/OAuthAuthorizeToken?oauth_token=\
    4%xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx&hd=default then hit enter.

One has to copy the above link to another machine (with gui browser). Next one has to grant access and then go back to the terminal and hit enter to complete the authorization.

BTW the access token is stored in:

~/.local/share/googlecl/access_tok_USERNAME

If access is granted from some machine it will probably suffice to copy the access token file to another machine to grant access for it as well (not tested).

The configuration file for googleCL is in:

~/.config/googlecl

If access is grated successfully the message similar to the following will be displayed:

Event created: http://www.google.com/calendar/event?eid=anFsdGFucXFwdmZnbW1sMmcwc2t1NDI3cm8gbG9vc2VoZWFkcHJvcDFAbQ

url | Fri, 22/03/2013 15:14 | tagi: english, google, googlecl, linux

How to import content from Blosxom to google Blogger

I have decided to give a try to Google Blogger service. I am an old dinosaur used to command line and tired with mouse and menus but as there is GoogleCL I am not scare. The problem is with my old posts---there is no way to post backdated blog entries with GoogleCL. A problem...

Fortunately there is export/import features on Blogger: one can backup blog content and/or upload it back to Google. In particular to import posts (and comments) into a blog, one have to click Import Blog from the blog's Settings. Next one have to select appropriate file and fill out the word verification beneath. The Blogger data format is Atom. So, to successfully import my old Blosxom entries I have to convert them to Atom.

I have made a few test entries and export them to check how the data looks like. Pretty wired but most of the content is irrelevant as it is concerned with formatting (css styles and such stuff is included). Also as I had comments disabled at my previous blog the problem is further simplified.

I have consulted Atom schema and tried with the following:

<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" 
      xmlns:openSearch="http://a9.com/-/spec/opensearchrss/1.0/" 
      xmlns:georss="http://www.georss.org/georss" 
      xmlns:gd="http://schemas.google.com/g/2005" 
      xmlns:thr="http://purl.org/syndication/thread/1.0">';

<id>tag:blogger.com,1999:blog-1928418645181504144.archive</id>
<updated>2011-10-22T12:34:14.746-07:00</updated>
<title type='text'>pinkaccordions.blogspot.com</title>
<generator version='7.00' uri='http://www.blogger.com'>Blogger</generator>

The meaning of the elements should be obvious. The last element (generator) is required by Blogger import facility, otherwise error message is returned.

According to the schema inside feed element there is zero or more entry elements:

<entry>
<id>ID</id>
<published>DATE</published>
<updated>DATE</updated>
<category scheme="http://schemas.google.com/g/2005#kind" term="http://schemas.google.com/blogger/2008/kind#post"/>
  
<!-- tags, each as value of attribute `term' of element category -->
<category scheme='http://www.blogger.com/atom/ns#' term='tag1'/>
<category scheme='http://www.blogger.com/atom/ns#' term='tag2'/>
<title type='text'>title</title>
<content type='html'>post content ... </content>
</entry>

There is a final </feed> to guarantee that XML file is well formatted.

I have assumed the only important feature of id element is that it's content should be unique. I have decided to use MD5sum of the post content as IDs to guarantee that.

Finally, my old Blosxom-compatible entries looks similar to the example below:

<?xml version='1.0' encoding='iso-8859-2' ?>
<html xmlns="http://www.w3.org/1999/xhtml"><head>
<title>Przed finałami RWC 2011</title>
<!-- Tags: rwc2011,rugby,francja,polsat-->
</head><body><!-- ##Published : 2011-10-20T07:20:26CEST ##-->


<p>W RWC 2011 zostały już tylko dwa mecze: jutro (piątek), o trzecie miejsce oraz

So it was extremly easy to extract title, tags and publication date and format Atom-compliant XML file with the following Perl script:

#!/usr/bin/perl
# Variant of Blosxom to Blogger conversion
# 2011/10 t.przechlewski
#
use Digest::MD5 qw(md5_hex);

print '<?xml version="1.0" encoding="UTF-8"?>
<!-- id, title/updated jest wymagane w elementach feed/entry reszta opcjonalna -->
<!-- wyglada na minimalne oznakowanie -->
<feed xmlns="http://www.w3.org/2005/Atom" 
      xmlns:openSearch="http://a9.com/-/spec/opensearchrss/1.0/" 
      xmlns:georss="http://www.georss.org/georss" 
      xmlns:gd="http://schemas.google.com/g/2005" 
      xmlns:thr="http://purl.org/syndication/thread/1.0">';

print "<id>tag:blogger.com,1999:blog-1928418645181504144.archive</id>";
print "<updated>2011-10-22T12:34:14.746-07:00</updated>";
print "<title type='text'>pinkaccordions.blogspot.com</title>";
print "<generator version='7.00' uri='http://www.blogger.com'>Blogger</generator>\n";

foreach $post_file (@ARGV) {

  my $post_title = $post_content = $md5sum = $published = '';
  my @post_kws = ();
  my $body = $in_pre  = 0;
  my $rel_URLs = 0;

  print STDERR "\n$post_file opened!\n";
  open POST, "$post_file" || die "*** cannot open $post_file ***\n";

  while (<POST>) {
    chomp();

    if (/<title>(.+)<\/title>/) {$post_title = $1 ; next ; }
    if (/<!--[ \t]*Tags:[ \t]*(.+)[ \t]*-->/) {$tags = $1 ; next ; }

    if (/<\/head><body>/) { 
      $body = 1 ; 
      ## </head><body><!-- ##Published : 2011-10-20T07:20:26CEST ##-->
      if (/##Published[ \t]+:[ \t]+([0-9T\-\:]+).+##/) { $published = $1; }
      print STDERR "Published: $published\n";
      next;
    }

    if (/<\/body><\/html>/) { $body = 0 ; next }

    if ( $body ) { 
      ## sprawdzam `przenosnosc URLi':
      if (/src[ \t]*=/) { 
	if (/pinkaccordions.homelinux.org/ || !(/http:\/\// ) ) { $rel_URLs = 1;  }
      }
      ## zawartość pre nie powinna być składana w jednym wierszu:
      if (/<pre>/)   { $in_pre = 1; $post_content .= "$_\n"; next ; }
      if (/<\/pre>/) { $in_pre = 0; $post_content .= "$_ "; next ; }
      if ( $in_pre ) { $post_content .= "$_\n"; }
      else {
	$post_content .= "$_ "; # ** musi być ze spacją **
      }
    }
  }

### ### ###

  if ($published eq '') { 
    warn "*** something wrong with: $post_file. Not published? Skipping....\n" ;
    close(POST);
    next ; 
  }
  if ( $tags eq '' || $post_title eq '' ) { 
    die "*** something wrong with: $post_file (tags: $tags/title: $post_title)\n"; }
  if ($rel_URLs) { die "*** suspicious relative URIs: $post_file\n"; }

  $post_content =~ s/\&/&amp;/g;
  $post_content =~ s/</&lt;/g;
  $post_content =~ s/>/&gt;/g;

  print STDERR "Title: $post_title Tags: $tags\n";

  @post_kws = split /,/, $tags;
  $md5sum = md5_hex($post_content);
  print STDERR "MD5sum: $md5sum\n";

  print "<entry>";
  print "<id>tag:blogger.com,1999:post-$md5sum</id>";
  print "<published>$published</published>";
  print "<updated>$published</updated>";
  print '<category scheme="http://schemas.google.com/g/2005#kind" term="http://schemas.google.com/blogger/2008/kind#post"/>';
  
  ## tags:
  foreach $k (@post_kws) { print "<category scheme='http://www.blogger.com/atom/ns#' term='$k'/>"; }

  print "<title type='text'>$post_title</title>";
  print "<content type='html'>$post_content</content></entry>";

  close(POST);

}

print "</feed>";

The minor problem was the default formatting of <pre>...</pre> which I use to show code snippets. I have to preserve line breaks (cf. $in_pre in the above Perl script) of pre element content as well as have to add the following to the default CSS styles (it is possible to modify CSS via Project →Template Designer →Advanced →Add CSS¹)

pre { white-space:nowrap; font-size: 80%;  }

To convert simply run script as follows:

perl blogspot-import.pl post1 post2 post3.... > converted-posts.xml

The above described script can be downloaded from here.

¹In Polish: Projekt →Projektant szablonów →Advanced →Dodaj Arkusz CSS

url | Mon, 24/10/2011 18:50 | tagi: atom.xml, blogger, blogging, blosxom, googlecl, perl