Skip, Take and Twitter Paging: Using TweetSharp To Download All Tweets from the Twitter Timeline

February 4 2010

I have been using TweetSharp lately and I am pretty happy with it so far. One thing that was a little confusing is how they implemented paging in their API on top of the Twitter API. 

What they did was to use the LINQ syntax (Skip and Take). Basically, think of Skip as the page number and Take as the number of records on a page. So, let's say you want to download all of a person's tweets. This assumes of course that you've already gone through the OAuth process and have permission to do so.  Twitter lets you download a maximum of 3200 tweets from a user, 200 tweets at a time.

So, the code to grab all of a user's tweets using TweetSharp would be as follows:

            for (int i = 1; i < 17; i++)
            {
                var query = FluentTwitter.CreateRequest()
                       .AuthenticateWith(_consumerKey,
                                         _consumerSecret,
                                         _accessToken,
                                         _accessKey)
                       .Statuses().OnUserTimeline().Take(200).Skip(i).AsXml();
                var responses = query.Request().AsStatuses();
                if (responses.Count() > 0)
                {
		    //do something with tweets
                }
                else
                {
                    break;
                }
	     } 

 So, what's going on here?  Basically I am attempting to download all 3200 tweets with a for loop, asking to take 200 tweets for each page, which is my counter. If I get less than 3200, I exit the loop, thus not unnecessarily calling the Twitter API (and increasing my chances of getting rate limited).

 

 

Putting Files at the Root of Blog Storage in Windows Azure

February 3 2010

Just learned today that by default you can't store files at the root in Windows Azure storage. So, what if you have http://foo.blob.core.windows.net which you've mapped to http://blob.foo.com and you want to put a file so that it resolves to to http://blob.foo.com/bar.png? Well you can create a special container called $root, which will map to the root.  If you do that, anything in the $root container will resolve as expected, so that http://blob.foo.com/bar.png would work.

If you are using Cloud Storage Studio, their UI supports this with the New Root Container option when creating containers.

 

Critical Hot Fix for Using Silverlight, WCF and Windows Azure

February 1 2010

I innocently added a Silverlight-enable WCF Service to my Azure application and then innocently generated a Service Reference for my Silverlight client. Everything worked fine on my machine; nothing worked once I uploaded it to Azure.  After going down a lot of wrong turns in figuring out what was wrong, I was able to use Fiddler to discover the following response from the service:

The message with To 'http://.../Service.svc' cannot be processed at the receiver, due to an AddressFilter mismatch at the EndpointDispatcher. Check that the sender and receiver's EndpointAddresses agree.

This led me to the following hot fix: http://code.msdn.microsoft.com/KB971842.  After installing the hot fix, in order to get things working, I had to both add the following to my serviceBehaviors behavior:

          <useRequestHeadersForMetadataAddress>
            <defaultPorts>
              <add scheme="http" port="81" />
              <add scheme="https" port="444" />
            </defaultPorts>
          </useRequestHeadersForMetadataAddress>

And then I also had to add the following attribute to my service:

[ServiceBehavior(AddressFilterMode = AddressFilterMode.Any)]

I then regenerated my proxy and, hooray, everything worked. More on this and other issues when using WCF with Windows Azure is explained here: http://code.msdn.microsoft.com/wcfazure/Wiki/View.aspx?title=KnownIssues 

 

 

The Twitter API: The Schrodinger's Cat of Web Service APIs

January 25 2010

I just started reading Professional Twitter Development: With Examples in .NET 3.5 (Wrox Programmer to Programmer) and came across this quote, which cracked me up:

The Twitter API is the Schrödinger’s cat of web service APIs. Until you call it, you never know if it's alive or dead. Sometimes the mere act of calling it is enough to kill it.
--Scott Koon,Witty

Hilarious. And true.

The Importance of Using CNames with Windows Azure

January 22 2010

I recently had to move an Azure service from one account to another.  Unfortunately, they could not migrate my account, which meant that I manually had to migrate it.  In the case of the service, I had created a cname {name}.mysite.com that pointed to the Azure DNS name ({name}.cloudapp.net).  Lucky me: I was able to migrate the service and then repoint the CName to the same DNS entry with no changes or problems.  But in the case of my blob storage ({name}.blob.core.windows.net) I hadn't created a Cname, which means I manually have to update any reference to anything in blob storage. 

Lesson learned: use CNames for everything in Azure in case you need to transfer accounts, ownership, etc.

Using Cerebrata's Cloud Storage Studio To Download IIS logs from Multiple Instances of Windows Azure

December 18 2009

I've written before about the most excellent tool Cloud Storage Studio by Cerebrata. I use exclusively for uploading blobs to Windows Azure as well as downloading my IIS log files. I just hit an issue as far as IIS logs which (fortunately) easily resolved. By default, Cloud Storage Studio does not preseve path settings when downloading files.  This presents a problem if you have multiple instances of your application on Windows Azure, because, when you go to download the log files, there will be duplicate log file names (aka  "u_ex09121403.log") from your different instances. 

The fix is easy. Go into options and see if the "Preserve directory path information in blob name when downloading files & directories" setting checked or not? You can check this setting either under "Configuration Settings --> Container/Blob Setting" or under "Download Window --> Download Settings" By default it is unchecked. If it is checked, then Cloud Storage Studio will preserve the folder hierarchy. You may need to restart the app after you set it as it didn't pick up my configuration changes after I set it. (They need to implement INotifyPropertyChanged!)

 

Incarnate

December 16 2009

 Just shipped Incarnate, a new lab from the fresh new looking Mix Online site. I think the appropriate tagline for Incarnate is "Who do you want to be today" as people are quickly realizing that it can be more entertaining to be a celebrity avatar than yourself.

Now you know what all those posts about JSON-P, Windows Azure, REST Starter Kit and more were about! 

And, as you may notice, I've implemented Incarnate on this blog, inside .NET Blog Engine. Unfortunately, I couldn't use their plug-in model and I actually had to modify the source, but it wasn't too bad. If you are running .NET Blog Engine and want to know how to install Incarnate into it, let me know.

overflow:hidden -- a fix to a CSS problem

December 11 2009

Just resolved a funky little css issue where I was having problems with getting Firefox, Chrome and Safari to respect the background-color I was setting. Turns out the div itself needed to have overflow:hidden added to its CSS; IE8 was interpolating that space for me.

Adventures With Windows Azure Diagnostics

December 4 2009

New blog post published up on Mix Online about Windows Azure Diagnostics -- my experience, gotchas, etc.

JSON-P: An Elegant Hack

December 2 2009

I've been doing a lot more blogging these days over at Mix Online. My most recent post was about JSON-P and, in particular, how to get it working with the WCF REST Starter Kit.

VSAchievements
Visual Studio Achievements
Karsten Januszewski (207 Points)