Yabfog

Via Stack Overflow.

Add, of course, git add the empty directory.

So helpful.

I have a Node.js program running as a daemon on a Linux VPS. Periodically, it polls a list of URLs using request. When it first starts, everything runs smoothly. But after running for a while, it starts getting 400 errors, and the longer it runs, the more URLs return 400 errors.

I could not understand what was going on. My code was basically structured like this:

Given that code, we know the req object is initialized with each function call. So, how could this script degrade over time?

Well, I finally tracked it down: COOKIES!

Yup, request has cookies enabled by default. So, I think what was happening was that cookies were being set (presumably, top domain-level cookies having the same name at different URLs or subdomains on the same domain) but the values in request's cookie jar were not being returned properly. That means the remote host was getting invalid cookies -- hence the 400 response for a "Bad Request."

I haven't yet spent the time to figure out if this is a bug in request. It's on my TODO list.

In the meantime, I've disabled cookies in the req object:
var req = { url: url, timeout: options.timeout, jar: false };

It's now working as expected.

A couple of fun things I've been working on to learn some new programming skills, namely node.js and MongoDB.

1. News Bit -- Remember Share Your OPML? Me too!

2. Linkblog -- A super easy linkblog tool.

I plan to open source the linkblog code soon. It also uses a shorturl tool I'm working on -- shorturl is open source. I still need to hook the hit stats (collected by the shorturl tool) into the linkblog tool, but other than that the tools work well!

Some time ago, I answered another user's question on Stack Overflow about database design for a multi-user feed aggregator. I also received an email from a developer asking for additional input, which I shared. But I thought I should put my response here, as well, for posterity's sake if nothing else. Note that my comments here assume MySQL as the database but should apply to any SQL database.

Basically, my emailer asked what to do about the fact that the posts table will get huge very quickly if we have multiple users and a row for each post for each user. It's actually a pretty basic relational database scenario, but if you start your project as a single user application and later decide it's going to be multi-user, you may not realize that you probably need to completely redesign your database.

So I've posted the schemae I use for my multi-user feed aggregator (a private project):

I've also posted a sql command that you can run in a cron job to remove read posts that are more than 14 days old:

As an aside, I'm amazed by how many people are writing new aggregators. Is it a common programming class exercise to write a feed aggregator or something?

Stop 0x00000024

Oy.

1. Grab Windows XP Installation CD.
2. Boot into Recovery Console
3. dir c: // cannot read directory.
4. chkdsk c: /p // 1 errors found
5. Reboot into Recovery Console
6. dir c: // success!
7. fixboot
8. Reboot into Windows // success

Woot!

I wrote this script some time ago. It's been working flawlessly for me, so I thought I'd share it here. It could use some progress messages, I suppose.


#!/bin/bash
DIR= # Put the file system path to your WordPress installation here. E.g., /var/www/html/blog
TMPDIR=$HOME/tmp
WPDIR=$TMPDIR/wordpress
cd $TMPDIR
rm -rf latest.zip ./wordpress # Clean up from the last run
wget -nd http://wordpress.org/latest.zip
unzip latest.zip
mv $DIR/wp-config.php $DIR/.config # Stash your configuration someplace safe
rm $DIR/*.{txt,html,php} # Delete the old install
rm -rf $DIR/{wp-admin,wp-includes} # Delete more. Don't delete plugins or themes.
cp -aR $WPDIR/* $DIR/
mv $DIR/.config $DIR/wp-config.php # Restore the configuration
# You may not need the last two lines. I like to give my web server the ability to write files.
chown -R .www-data $DIR/*.php $DIR/wp-admin $DIR/wp-includes
chmod -R g+w $DIR/*.php $DIR/wp-includes


In case I need to remember that I did this...

By default, the Mac Terminal assigns the "Clear Scrollback" command to Command-K. This is annoying, because I often use Nano, which uses Control-K to cut a line of text, and I frequently press Command-K instead of Control-K by accident. This clears my screen while I'm in the middle of editing a file. Very annoying.

The solution is to assign a new (harder to hit by accident) command to the "Clear Scrollback" command, so that when I inevitably mistype Command-K, nothing happens.

I decided to install Dave Winer's River2 to supplement my usual feed reading. Now that I can access it via its smart use of Dropbox, it should be good for feeds that I don't feel like I need to see every headline.

One of the things I love about River2 is that it's an app that runs in the OPML Editor, which means that it is endlessly hackable and (apropos to this post) you can fix your own bugs.

So here's a bug report. And fix. (Actually, it could be a workaround for a bug in another application, as I explain below).

1. What I was doing: From the Tools > River2 > Pages menu, I selected a page to view (any one, it's the same bug no matter which page).
2. What I expected to happen: I expected the selected page to open in my default web browser, Pale Moon (a Windows-optimized build of Firefox)
3. What actually happened: Nothing. Not even an error dialog.

I immediately suspected that the problem was the communication between the OPML Editor and the Pale Moon browser. After all, there was a major bug for the longest time in Firefox's DDE implementation that required a workaround.

Bottom line: the OPML Editor's DDE implementation expects that the DDE service name is the same as the name of the executable with the filename suffix removed. So, for Excel, the service name is "excel," and for Firefox it's "firefox." But the service name is determined by the application, and the Pale Moon developers decided that its service name would be "Pale Moon," not "palemoon." A simple patch to system.verbs.builtins.webBrowser.openURL resolves the problem.

if string.lower (id) contains "palemoon" { // 2/11/11; 12:09:06 AM by DJM
	ddeName = "Pale Moon";
	return (webBrowser.callBrowser (ddeName, "WWW_OpenURL", s+",,0,0,,,,"))}

The function webBrowser.callBrowser expects ddeName to be the name of the executable, from which it attempts to remove the ".exe" suffix. Luckily, if the function is passed any string without an ".exe" suffix, it just accepts the passed string as the DDE service name.

Here's the full context:

That ",,0,0,,,," nonsense is part of the DDE message that Pale Moon expects: