Proof that Netflix is the only thing holding the USPS afloat

By Patrick | Published: March 3, 2012

"Netflix Only" chute at the post office

Posted in Uncategorized | Leave a comment

LivingSocial/Fiesta Americana Resort Bait & Switch Scam

By Patrick | Published: December 1, 2011
UPDATE: I contacted LivingSocial about this issue, and they responded in a timely manner, and offered a partial refund, which I feels remedies the situation.  I am leaving this post up to tell the story, and to alert people that they should always be on guard for issues like this, but at least LivingSocial has made a good effort to rectify the situation.
My wife and I recently returned from a trip to Los Cabos, Mexico, where we stayed at the Fiesta Americana Grand Los Cabos Golf & Spa Resort.  We purchased the resort package through “Living Social Escapes” (archived as PDF in case Living Social removes the page).  As excellent of a time we had while enjoying the warm weather and the beautiful beach, some of the items included in the offer we felt were misleading and blatantly untrue.

Under one of the bullets in the “escape kit” it states: “Daily Continental Breakfast, Afternoon Cocktails, and Hors D’œuvres at the Grand Club and Free Upgrade to Grand Club Level for First 50 Vouchers Sold”.  This gives the impression that daily continental breakfast, afternoon cocktails and hors D’oeuvres are included in the package deal and that the first 50 vouchers sold will receive free upgrade to Grand Club Level.  After being denied the continental breakfast, we spoke to the front desk.  The front desk people read it as “all of the items in this statement are only for the first 50 people.  This is a very tricky and misleading statement. Furthermore, in the body of the webpage, it states

“You’ll know you’ve been good this year as you enjoy meet-and-greet services and private ground transportation from Los Cabos International Airport, private check-in and check-out service, daily continental breakfast, and a $600 resort experience credit.”

This states nothing about the “first 50 vochers sold”.  We probably would not have bought the package had we know that the continental breakfast was not included, and instead found a package that included the amenities we were looking for at the right price.

Under another bullet it states:  “In-Room Bottle of Wine and Canapés Daily”.  When we first arrived, there was a bottle of wine and a plate of canapés in our room waiting for us.  The reminder of the days, there was no wine or canapés.  When we asked the front desk about this, they told us that a bottle of wine was only supplied on the first day and that Living Social lied about that in their ad. One front desk woman even admitted that the statement was a bait and switch tactic.

Overall, we are very unimpressed with Living Social’s misleading tactics.  We would love to be able to take advantage of the wonderful offers they have available, however they have lost a lot of our trust.

Posted in Uncategorized | Tagged | Leave a comment

Accessing Erased Type Parameter Information in Scala

By Patrick | Published: November 2, 2011

One of the things holding scala back from being a more robust language is the fact that it runs on the JVM. (On the other hand, this is also one of its strengths–you can easily interoperate with existing Java code and libraries, as well as any other code that runs ont he JVM, like Groovy or Clojure.) Because Scala runs ont he JVM, it suffers from type erasure, which means that any generic type parameters that are in the code are lost after compilation, and are no longer present in the byte code, and thus at runtime. (This was a design decision made by the Java team when they introduced generics in Java 1.5, in order to preserve byte code backward-compatibility.)

In order to work around this, Scala introduced the Manifest class, which captures the type information during compilation, and allows you to access this type at runtime.  This was originally designed to allow for the creation of arrays of the generic type at runtime, but it can also be used for other custom generic types.  It uses implicit parameters to capture this information.

The problem (that I had), though is that you can’t seem to access a class’s type parameters directly in a method of that class.  So, this does’t work:

import scala.reflect.Manifest
class MyClass[T: Manifest] {
  def myType(implicit m: scala.reflect.Manifest[T]) = m.toString
}

You won’t get a compilation error, but you will just get ‘Nothing’ as your type inside your method:

scala> new MyClass[Long]
res17: MyClass[Long] = MyClass@2769aba2
scala> res15.myType
res18: java.lang.String = Nothing

This is because there was no type parameter to the method itself, and the [T] in the method’s implicit argument is not the same [T] that is used to parameterize the class.  If we add a type parameter to the ‘myType’ method, we can get closer to what we want:

class MyClass2[T: Manifest] {
  def myType[U](implicit m: scala.reflect.Manifest[U]) = m.toString
}

Then, we can try it out:

scala> new MyClass2[Long]
res19: MyClass2[Long] = MyClass2@6c6455ae
scala> res19.myType[Int]
res20: java.lang.String = Int

Okay, so now we are getting some type information out, but it really isn’t ideal.  Note that the type parameter inside the ‘myType’ method was ‘Int’, which was the parameter we passed to the method, not ‘Long’, which is what we passed to the class.  What if we want to get the class’s type parameter?  We certainly don’t want the calling code to have to keep track of the type parameters in multiple places (once at the constructor call, and once more at the method invocation).  So, we can add a second method to help smooth this out:

class MyClass3[T: Manifest] {
  def myType[U](implicit m: scala.reflect.Manifest[U]) = m.toString
  def myTypeWtihoutExtraParam = myType[T]
}

And, then, we get:

scala> new MyClass3[Long]
res30: MyClass3[Long] = MyClass3@543d8ee8
scala> res30.myTypeWtihoutExtraParam
res31: java.lang.String = Long

So, you can see that the type that was passed in to the class constructor as a generic parameter is now available in the method, without the method caller needing to supply it after the object is created.

A couple of things to note:

  • Don’t forget to import scala.reflect.Manifest
  • The type parameter in the constructor definition is context-scoped, so the ‘: Manifest’ part is important.  This technique won’t work if you just specify the parameter as ‘[T]‘.  If you leave this off, you will get an error that reads, “No Manifest available for T.”
Posted in computers | Tagged , , , | 1 Comment

US orders news blackout over crippled Nebraska Nuclear Plant– Coverup or Hoax?

By Patrick | Published: June 23, 2011

In the Morning, I came across an article from the Pakistani news source, The Nation, titled US orders news blackout over crippled Nebraska Nuclear Plant.  If this is true, it is terribly troubling–both because of the implications of a nuclear meltdown in the midwestern United States, as well as the idea of a government coverup to try to silence the news media from reporting on it.

However, the news article that is ‘reporting’ on it is of questionable origin.  The general tone of the article stinks of anti-US propaganda, with passages like this:

Obama’s fears of the American people turning against nuclear power, should its true dangers be known, appear to be valid as both Germany and Italy (whose people, unlike the Americans, have been told the truth) have turned against it after the disaster in Japan and vowed to close all of their atomic plants.

Parts of the article sound like it was just written by (presumably American) critics of Obama, since the entire article is blaming him for the coverup, and not the government in general.  Other parts, though, like the passage I quoted above, seem to have more of a general anit-US slant.

So, the author of this article clearly has an agenda, of some sort.  But, does that mean that it is entirely false?  Or is there some grain of truth to this whole thing?

From what I can tell, in the 5-10 minutes I just spent researching this:
  • There is indeed a 4-level scale for rating nuclear power plant ‘events’ in the United States.
  • Fort Calhoun Power Plant recently (June 8, 2011) had a small fire, which caused a partial evacuation, and triggered a level 2 ‘Alert’ event.
  • Fort Calhoun Power Plant issued a level 1 (the least-alarming type of event) event in connection with the flooding of the Missouri River.
  • The FAA has indeed issued a no-fly NOTAM order for the area.
  • There was an article titled “Low-level emergency declared at nuclear power plant” from JournalStar.com that was supposedly relating to this flooding alert.  The article is no longer at the URL linked from Wikipedia.  Does this indicate a coverup?  Maybe, maybe not.  Strangely, the retrieved date cited on Wikipedia is a while before the alert was issued by the NRC.  This makes me wonder how likely the link was to ever have been there.
  • There are a lot of random blogs, of varying degrees of crack-pot-itude talking about conspiracies, coverups, etc.  I don’t know if they are all just in the echo chamber, and this was all sparked by that one article from Pakistan, or if there is more to it.  There was an article on a Ron Paul-related blog, dailypaul.com linking to the Pakistani article, so it is definitely making its rounds.
Well, that’s all for now.  Lets hope the meltdown & coverup story is just a hoax.
Posted in Uncategorized | Tagged , , , , | Leave a comment

Mapping site should be faster now

By Patrick | Published: December 11, 2010

Over the past few weeks, I have been noticing that the site I created to allow users to plot multiple addresses on Google Maps would sometimes become very slow when a lot of people are using it. So, I started to investigate the cause.

(I am going to split this post into two sections, the first aimed at non-programmers, and the second at programmers, in case they are interested. I realize that most of the people that use my maps site probably don’t really care how it works under the hood, so the first part is for them. On the other hand, if I can share what I learned from this experience with other programmers, then they can read the whole post, focusing on the second half.)

Overview

The site uses a database to store the information about maps that people choose to save. I found that one of the database parameters was the most likely culprit for the slowness. Basically, it was optimized for a database that gets information read from it almost all the time, but when data is written to it, the whole thing locks, and doesn’t allow other people to read data from it while that write is happening.

So, now that I had found the likely cause of the issue, I started investigating how to fix it. I found that there were too methods, so I just picked the one that seemed easier. After a few hours of issuing the command to make this change, it appeared to still be working. Not sure if something was wrong, and the process stalled, I cancelled and re-issued the command. This was probably my biggest mistake. I later realized that the process just takes a long time, and I should have let it run. I basically did this a couple of times, trying the other method to make this change as well. Of course, this site is something I do in my free time, so I had to do other things during this process as well, such as go to work, sleep, and eat. In the end, once I realized that the process was working, and I should just let it run, it took about 10.5 hours.

Long story short, the site is back up, and hopefully it will be faster from here on out.

Technical Details

The parameter that I was trying to change was the MySQL storage engine. Originally I had it set as MyISAM, most likely because that was the default when I created the database. MyISAM is extremely fast when you are doing just reads from a table, but when you do a write, it will lock the entire table, preventing any other threads from access it for reading or writing. I suspect this is what was causing the performance issues with the site.

So, I decided to change it to the InnoDB storage engine. InnoDB is still fast for reads, but not as fast as MyISAM. One of the big advantages to InnoDB, though, is that it uses row-level locking, instead of table-level locking. So, if one request is adding a new address to the table, another request can be reading some other addresses. (The one that is currently being written won’t be able to be read, but it is very unlikely that someone would be trying to view a map that has not been fully created yet.)

First Approach

So, there are two main approaches to changing the database storage engine. The first one I tried, which seemed simpler at the time, was to issue the following command:

ALTER TABLE tablename ENGINE = InnoDB

I found this on Farhan Mashraqi’s site during my research, so I decided to try it first (after making a backup to the database, of course).

I restored the backup on my laptop, and decided to give it a test run. I tried it on some of the smaller tables, and it seemed to work. Since I wanted to get the upgrade done quickly, and get on with my life, I didn’t try upgrading the whole database on my laptop, and just started doing it on the production server.

This was really my biggest mistake. I should have done the whole upgrade on my laptop, start to finish, before trying to change anything on the real site. (I don’t really know why I did this; I never would have been so sloppy with a similar task at work.)

So, as I said, I started making the change on the real site. After a could of hours, it still wasn’t finished, and I noticed that in the MySQL Administrator GUI, on the Catalogs tab, the value in the ‘rows’ column was jumping all over the place. It would go up, down, and change in multiple different tables. This didn’t seem to make any sense, and led me to believe that something was wrong, and that perhaps I should restart the process. As it turns out, the row count is just bogus, and shouldn’t be trusted. But, I didn’t realize that at the time, so I restarted the process, and went to bed.

Second Approach

In the morning, I checked on it, and it was still chugging along. I figured that something must be wrong, since it was taking so long, and the row counts were still acting weirdly (I didn’t know they were bogus yet). So, I decided to try the other method of changing the storage engine, which is to take your backup file form mysqldump, and change the parts where it says ‘MyISAM’ to ‘InnoDB’, and then restore it. I decided to do this with the sed command, since this backup file is rather large, and I wasn’t sure if vim would choke on it. The following command did the trick:

sed s/MyISAM/InnoDB/ backupfile.sql > updatedbackupfile.sql

Then, I just ran that SQL script. It also took forever. I thought that perhaps the issue was that my site runs on a virtual server (from slicehost, and overall I am very happy with their service), this means that the disk i/o can sometimes be slow. It generally isn’t a problem, but it is one of the disadvantages to a virtualized server. Since what I was doing was very i/o intensive—reading a large file, doing a little bit of processing, and then writing the data out into the database, which resides on disk—I thought that perhaps this was the culprit. So, once again, I canceled the process.

The next (and final) try, I did the same thing, except I had the modified backup file that I was trying to restore on my laptop, and I restored it over the network. This meant that the server, with its virtualized disks, was only responsible for writing the data to disk, and not reading it from disk. It read the data from the network. I also learned that the row count in the MySQL Administrator GUI was bogus, and I shouldn’t worry myself with it.

A little more than 10 hours later, the database is back up and running, and I removed the maintenance message from the site!

Lessons Learned

  • Always do something first in a test environment before touching the production environment
  • Database actions that change entire large tables take a long time. Let them run
  • Always do something first in a test environment before touching the production environment
  • The row count in the MySQL Administrator GUI is bogus. Don’t even look at it.
  • Always do something first in a test environment before touching the production environment
Posted in computers, gmaps.kaeding.name | Tagged , , , | 29 Comments

New Homebrew Question/Answer site

By Patrick | Published: November 19, 2010

I’ve been active on the programming question & answer site, StackOverflow.com for a few years now, and in the past several months, the company behind that wonderful resource has branched out into other topics.  This week, the new Homebrew.StackExchange.com site launched in public beta.

The software this site runs on is finely tuned for asking questions, and getting useful answers quickly.  It is specifically designed not to be used for discussion, since other forum software handles discussions relatively well (but is really cumbersome for asking questions, and evaluating which answer is the most helpful).

So, the way the site works is:

  • You ask your question.  You provide a title for the question, and then go into as much detail as you feel is necessary in the body of the question.  Of course, the more detail you can provide, the easier it will be for people to understand both what you are asking, and why you are asking it, so they can give you a useful answer (or answers :) )
  • People see your question, and post comments asking for clarification, if they feel it is needed.  You can then edit your question to add more details.  This is important, because then if people find this page later, because they have a similar question, all the relevant details are in one place, and there is no need to sift through 12 pages of forum posts trying to understand the problem and solution.
  • People post answers to your question.  Again, they should be posting as much detail as possible, to try to give you a useful solution to your problem.
  • People will read your question and the posted answers, and vote on each, indicating how helpful they feel each answer is, and also on how valuable the question is.
  • You will read the answers posted, and post comments on them if you need more details (just like the people posted comments on your question asking for clarification, you can do the same if the answer is unclear).  You can also vote each answer either up or down, based on whether you think it is helpful or not.
  • You can then mark one of the answers as ‘accepted’, to indicate that it solved your problem.  This servers as a ‘thank you’ to the answerer, and closes the feedback loop.


profile for pkaeding at Homebrew, Q&A for dedicated home brewers and serious enthusiasts
All of this voting will build up ‘reputation’ points for the users of the site.   To the left is my reputation badge on the new site. As you gain reputation, the system ‘trusts’ you more, and gives you access to more tools to help maintain the site.  The site is very much driven by the community, and really does a great job of building and nurturing the community.

Also, all of the data gathered by the site (questions, answers, etc) is released in a monthly data dump under a Creative Commons license, so you don’t have to worry about the site taking all of this content, and then turning it into a for-pay site, or anything.  If that happened, you could just take the data dump, and start your own site with it.

Posted in homebrewing/beer | Tagged , , | Leave a comment

Prioritizing Netflix traffic with DD-WRT

By Patrick | Published: November 15, 2010

Network traffic shaping is an interesting topic, that allows you to ensure that certain traffic gets priority over other traffic.  When applied at the ISP level, this can get controversial, as you start getting into Network-Neutrality issues (where one company’s traffic gets priority over another company’s, which could lead to large media corporations silencing grassroots communication).

At the local network level, though, it means that you can ensure that certain traffic (like streaming Netflix videos) won’t be slowed down just because other, less important traffic (like an off-site backup job), is also flowing through your WAN connection.  DD-WRT makes all of this possible (and not too difficult) on the NAT/QoS->QoS tab.

NAT/QoS->QoS

In the first section, titled ‘Quality of Service (QoS)’, set the following options:

  • Start QoS => Enable
  • Port => WAN
  • Packet Scheduler => HTB
  • Uplink => (whatever your ISP gives you for an uplink speed)
  • Downlink => (whatever your ISP gives you for an downlink speed)

You may want to check out speedtest.net or a similar service to see what your uplink and downlink speeds are.  If you can get this information from your ISP, that would be better, since the more accurate these values are, the better this will work.  If you enter too high of a value, the shaping won’t kick in because the router will think that it has more bandwidth to paly with.  If you choose too low of a value, you will end up wasting bandwidth, and your router will not use it all.

Now, if you only ever watch Netflix from a device that won’t be sending low-prioirty traffic as well (such as a Wii or Roku box), you can just enter that device’s MAC address in the MAC Priority section.  Add the MAC address(es) and then select ‘Premium’ for the priority.

MAC Priority Settings

On the other hand, if you have a home server connected to your television, and you use this both as a file server (which runs off-site backup jobs to ensure your data is not lost in the event of fire, burglary, or other catastrophe) and as a media player, you will want more fine-grained control, since not all of the traffic to that device will have the same priority.

So, we will need to set up some Netmask Priority rules.  This will give traffic to/from Netflix a higher than normal priority.  Inspired by Jonathan Kamens, I first set my offsite backup (to Amazon’s S3 service) a lower than average priority.  Then, I followed the same approach to itentify the subnet used by Netflix to stream their movies.

Using Little Snitch, I learned that Netflix uses LibSyn’s content-delivery network to stream the data.  Specifically, I noticed a lot of traffic coming from netflix-380.vo.llnwd.net.  Now, that server alone is not enough, because no doubt every time you connect, you will get a different server int he pool (like netflix-379…, netflix-381… etc).  So, I got the IP address for this server using the ‘ping’ command:

[pkaeding@tripel:~] 22:35:43
% ping netflix-380.vo.llnwd.net
PING netflix-380.vo.llnwd.net (208.111.173.130): 56 data bytes
64 bytes from 208.111.173.130: icmp_seq=0 ttl=56 time=15.002 ms
64 bytes from 208.111.173.130: icmp_seq=1 ttl=56 time=16.956 ms
64 bytes from 208.111.173.130: icmp_seq=2 ttl=56 time=14.714 ms

Now that we have the IP address (208.111.173.130), we need to know what block of IP assignment it belongs to.  IP addresses are assigned to companies in blocks, so it is a good bet that we want to prioritize all traffic to that network in the same way.  The ‘whois’ command will help us learn this information:

[pkaeding@tripel:~] 20:26:16
% whois 208.111.173.130
GeekTools Whois Proxy v5.0.5 Ready.
Checking access for 67.174.196.72... ok.
Final results obtained from whois.arin.net.
Results:
#
# The following results may also be obtained via:
# http://whois.arin.net/rest/nets;q=208.111.173.130?showDetails=true&showARIN=false
#

NetRange:       208.111.128.0 - 208.111.191.255
CIDR:           208.111.128.0/18
OriginAS:       AS22822
NetName:        LLNW-3
...

What we are interested in is the ‘CIDR’ field.  This is what refers to the block of IP Addresses that we are trying to prioritize.  Go back to DD-WRT, and in the ‘Netmask Priority’ section, add an entry for this network.  Then, assign it to the ‘Premium’ priority.  (In the screenshot, you can see that I have the S3 network set to ‘Bulk’ as well as the Netflix traffic set to ‘Premium’.

Netmask Priority Settings

In the end, I can watch movies while my 130GB photo collection is copied to Amazon’s cloud service!

Posted in computers | Tagged , , , | 3 Comments