Sqoop hive import fails with IncompatibleClassChangeError

Exception in thread “main” java.lang.IncompatibleClassChangeError: class
com.facebook.fb303.FacebookService$Client has interface org.apache.thrift.TServiceClient as super class

As with most obscuro Hadoop issues, I found absolutely no documentation about this error anywhere on the interwebs. It turns out, this is because no one else had a sqoop install as hosed as ours 🙂

For some reason our sqoop installation had some random thrift jars lying around in /usr/lib/sqoop/lib, including:

# ls -l /usr/lib/sqoop/lib/*thrift*
/usr/lib/sqoop/lib/hadoop-thriftfs-0.20.2-cdh3u2.jar
/usr/lib/sqoop/lib/thrift-0.2.0.jar
/usr/lib/sqoop/lib/thrift-0.5.0.jar

I have absolutely no idea why, but maybe someone just copied stuff directly from a cdh3 installation or hacked around a previous problem. No idea. Needless to say, that’s a lot of unnecessary thrift crap, especially when none of those jars match up with the version of thrift that Hive is using (libthrift-0.7.0.jar).

So, if you happen to inherit a crazy hacked up CDH4 installation like I did, you can fix it with:

# rm /usr/lib/sqoop/lib/*thrift*

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s