<html>

<head></head>

<body>
    <div class="item">
        <div class="item-header">



        </div>

        <div class="item-body">

            In recent weeks, it seems like I spent most of my time finding bugs in the <a href="https://apiwiki.twitter.com">twitter API</a> or third party libraries. Maybe I should get a job in their testing team. This latest bug was discovered in the twitter search API. The <em>from_user</em> and the <em>from_user_id</em> fields do not quite match. Here are some examples:
            <p style="padding-left: 30px;"><span style="color: #993300;">{"profile_image_url":"http://a1.twimg.com/profile_images/311054570/SecuritySystemPlaza.comLogo_normal.jpg","created_at":"Sun, 29 Nov 2009 12:39:51 +0000","from_user":"SecurityPlaza","to_user_id":null,"text":"GE SECURITY - USA TX6010011 ONLY
                    $78.66 AT WWW.SECURITYSYSTEMPLAZA.COM","id":6168853273,"from_user_id":28463195,"geo":null,"iso_language_code":"en","source":"&amp;lt;a href=&amp;quot;http://apiwiki.
                    twitter.com/&amp;quot; rel=&amp;quot;nofollow&amp;quot;&amp;gt;API&amp;lt;/a&amp;gt;"}</span>

                Here the from_user field is given as SecurityPlaza and the from_user_id field shows a value of 28463195, if you look at http://twitter.com/users/28463195.xml you will see that the user id of 28463195 corresponds to <a href="https://twitter.com/ballercoolkid">@ballercoolkid</a> on the other hand http://twitter.com/users/show.xml?screen_name=SecurityPlaza&nbsp; gives a user id of 56297110 to &lt;a href="http://twitter.com/SecurityPlaza"&gt;@SecurityPlaza&lt;/a&gt;. So which one of these users sent out the tweet? it was @SecurityPlaza

                Here is the second example:
            </p>
            <p style="padding-left: 30px;"><span style="color: #993300;">{"profile_image_url":"http://a1.twimg.com/profile_images/364463510/logo_normal.jpg","created_at":"Sun, 29 Nov 2009 09:32:56 +0000","from_user":"hindichannels","to_user_id":null,"text":"Sacked opposition party member to spearhead campaign against Genera
                    l Fonseka: Nov 29, Colombo: A member of the main... http://bit.ly/8OBKwS","id":6166287234,"from_user_id":32271894,"geo":null,"iso_language_code":"en","source
                    ":"&amp;lt;a href=&amp;quot;http://twitterfeed.com&amp;quot; rel=&amp;quot;nofollow&amp;quot;&amp;gt;twitterfeed&amp;lt;/a&amp;gt;"}</span>

                Here the user hindichannels has been associated with the userId of 32271894 where as that ID really belongs to <a href="https://twitter.com/mrsdraz">@mrsdraz</a> the correct id for <a href="https://twitter.com/hindichannels">@hindichannels</a> is 61683443 interestingly the results are easily reproducable. Two different searches produced the same errornous output ('<a href="https://search.twitter.com/search.json?q=colombo&amp;rpp=100&amp;show_user=true&amp;page=1">fonseka</a>' and '<a href="https://search.twitter.com/search.json?q=colombo&amp;rpp=100&amp;show_user=true&amp;page=1">colombo</a>')

                I tried these searches between 6:00-7:00pm on Nov 29th by the time that you try these web API calls on your own, the search results would have changed dramatically. So <a href="https://www.raditha.com/blog/archives/raditha.com/blog/images/twit-json-samples.tar.gz">here is a cached copy</a> for you. By the way, the search results may have contained other inaccuracies but these are the only ones that I noticed thanks to some safety checks we have in the <a href="http://pitupasa.com/">Pitupasa</a> code base.

                The XML search results does not have this problem due to the simple fact that the user ids are not included in it!


            </p>
        </div>
    </div>


</body>

</html>

Twitter Search API inconsistency

<html>

<head></head>

<body>
    <div class="item">
        <div class="item-header">



        </div>

        <div class="item-body">

            <p></p>
            <p>If you see a <a href="https://twitter.com/">Twitter</a> or Facebook app that asks for your username and password you should run a mile. Admittedly <a href="https://oauth.net/">OAuth</a> is not anymore secure than CHAP authentication but there is a crucial difference; most people use the same username/password (or at least the same password ) on multiple websites, email addresses, memory cards and what have you. By necessity when a web app saves your password for Twitter or Facebook or any other site, it has to be encrypted with a symmetric key, which means the plain text version can be easily obtained.</p>
            <p>Users don't need any special effort for Oauth but developers do. Writing an application that uses OAuth instead of passwords is supposed to take just a wee bit more effort, (but all seasoned programmers know what exactly that means).</p>
            <p>One of the issues we ran into was caused by Twitter returning an 'Incorrect signature' error for List Delete member api calls. The signature was being generated using the <a href="https://github.com/abraham/twitteroauth">TwitterOauthLib</a>. The signature was correct. It worked fine for all other methods excepts for DELETE calls. If you passed in the same parameters to a different Oauth library, you would get the exact same signature so are we seeing a bug in the twitter API?&nbsp; You might also suspect that this is because the underlying HTTP client library didn't support the DELETE HTTP method. In this case it's curl and the DELETE method is supported. Nevertheless I tried using an HTTP POST while sending the _method=DELETE parameter as suggested by the <a href="https://apiwiki.twitter.com/Twitter-REST-API-Method%3A-DELETE-list-members">Twitter API documentation</a> with out any luck.</p>
            <p>On further investigation I found that all the parameters were being posted for both DELETE and POST requests. <a href="https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html">RFC 2616</a> doesn't specify how you should pass parameters (request body or query string) for DELETE requests. I decided to change the TwitterOauth code to include the parameters as query strings (get the <a href="http://images.raditha.com/blog/images/oauth-patch.txt">patch</a>). Sure enough twitter accepted the request without the incorrect signature error.&nbsp; The modified code now looks like this:</p>
            <p></p>
            <p></p>

        </div>
    </div>


</body>

</html>

Solving a Twitter OAuth and Incorrect Signatures Mystery

<html>

<head></head>

<body>
    <div class="item">
        <div class="item-header">



        </div>

        <div class="item-body">

            <p style="text-align: left;">Upgrading Fedora is easy. It's the post upgrade clean up that kills you. As already mentioned, <a href="fedora-12-rant.html">the move from version 11 to 12,</a> left behind 581 packages from F11. These should ideally be removed to avoid unexpected segfaults, system crashes and inconsistent behavior. It shouldn't happen but it does happen, if you don't cleanup now, later you will have a hard time figuring out some programs will not start up or package updates cannot be applied.&nbsp; `package-cleanup --cleandupes` is a good starting point. This command will remove the older version from the system when there are two versions of the same package. Well that's what it's supp<span style="color: #000000;">osed to do, but it choked:</span></p>

            <p style="text-align: left; padding-left: 30px;"><span style="color: #ff0000;">service fuse does not support chkconfig
                    error: %preun(fuse-2.7.4-3.fc.x86_64) scriptlet failed, exit status 1</span>
            </p>
            <p style="text-align: left; padding-left: 30px;"></p>
            <p style="text-align: left;"><span style="color: #ff0000;"><span style="color: #000000;">Trying to manually remove the fuse package with yum gets you nowhere fast:</span></span></p>

            <p style="text-align: left; padding-left: 30px;"><span style="color: #ff0000;">Running Transaction
                    service fuse does not support chkconfig
                    Error in PREUN scriptlet in rpm package fuse
                    fuse-2.7.4-3.fc.x86_64 was supposed to be removed but is not!</span>
            </p>
            <p style="text-align: left;"></p>
            <p style="text-align: left;"><span style="color: #ff0000;"><span style="color: #000000;">Using `rpm -e` produces the exact same error as package-cleanup (which is because package-cleanup was merely passing through the error message it received from rpm). Fortunately the --noscripts paramter to rpm can be used to force the uninstall.</span></span></p>
            <p style="text-align: left; padding-left: 30px;">rpm -e --noscripts fuse-2.7.4-3.fc.x86_64 fuse-2.8.1-1.fc.x86_64</p>
            <p style="text-align: left;">After the rpm command was executed in this manner, package-cleanup also would run to completion with out any errors. But it has a funny idea about what completion is; there are still 433&nbsp; .fc rpms lying about. Most of these packages are from third party repos such as RPM fusion. A yum update ought to fix it and indeed the applying 684MB of updates resulted in all but 68 of the fedora 11 rpms disappearing from the system. Some of these are old kernels/kernels headers which can be removed with `package-cleanup --oldkernels` that still leaves around 60 rpms that you might have to manually uninstall and re install. You might also right the following simple command to remove them all en mass:</p>
            <p style="text-align: left; padding-left: 30px;">yum remove `rpm -qa | grep fc`</p>
            <p style="text-align: left;">The only catch is that it might remove some other important packages but putting those back in might be easier than removing 60 rpms one at a time. BTW why are these rpms tagged as FC11 or FC12, with FC standing for Fedora Core when the OS itself is no longer called Fedora Core? That name hasn't been used for ages. Now it's just Fedora</p>
            <p style="text-align: left;"></p>
            <p style="text-align: left;"></p>

            <p style="text-align: left;"><span style="color: #ff0000;">
                </span>

            </p>
        </div>
    </div>


</body>

</html>

Fedora 12 Post Install Cleanup.

<html>

<head></head>

<body>
    <div class="item">
        <div class="item-header">



        </div>

        <div class="item-body">

            <p></p>
            <p>It is said that thanks to Fedora's half yearly release cycle, you always get to use cutting edge technology. Bleeding edge would be more appropriate and it is no surprise that some RPM repositories do have a branch labeled 'bleeding'.</p>
            <p>Every Fedora upgrade introduces more problems than it solves which makes you wonder if this is any different from Windows. The new features are usually not even worthy of a blog post and <a href="the-never-ending-problems-with-fedora-updates..html">an upgrade never goes smoothly</a>. So why bother? Well each time I install a new version, I sweart that I will skip the next upgrade but after six months of using a buggy OS, you get tired of it and want to get those problems fixed the new version will usually do that but introduce a new set of problems.</p>
            <p>The major issue with the current version F11 is that X server keeps crashing for no apparent reason. Ctrl-Alt-backspace (annoyingly disabled in most new xorg versions, you need to enable it by editing the key mappings) will not kill the server. Ctrl-Alt F2 , F3 etc will not get you a console. Yet the machine is not dead. You can very easily SSH into it from another computer. You can even kill x from the shell but X refuses to die. Well it does die but the screen does not change. It remains steadfastly stuck on the last screen that you worked on. Even those annoying Ajax spinners are frozen for once. The only fix is to go all the way down to runlevel one and then do an init 5 from there. But who has the patience of all that stuff? I usually just hit the reboot button.</p>
            <p>So my desktop, which once upon a long ago would run continuously for weeks at a time, now needs to be restarted several times a day. One sad day, X crashed thrice. Surely even m$ can do better than that?</p>
            <p>If that's not bad enough the update makes things worse. X wouldn't start at all. There is absolutely nothing in the Xorg.log file either but it would start up when you type startx from the shell! explain that!!</p>
            <p>I suspect the reason has lots to do with F11 and F12 RPMS getting mixed up. Each Fedora installation leaves an unbelievable number of packages from the previous version lying around. This version has left 581 F11 rpms behind. The worst was when the first 64bit version came out. There were so many 32bit RPMs left behind that almost nothing would&nbsp; run, even the boot process got stuck.</p>
            <p>In the past I got <a href="1423.html">so fed up with fedora</a> that I even tried Ubuntu and Debian only to find that they are <a href="1458.html">even more annoying</a>.</p>
            <p></p>

        </div>
    </div>


</body>

</html>

Monthly Archives for 2009 11

Twitter Search API inconsistency

Solving a Twitter OAuth and Incorrect Signatures Mystery

Fedora 12 Post Install Cleanup.

Fedora 12 (Rant)