Following an investment guru a good idea?

The best answer here is "maybe, but probably not". A few quick reasons:

• Investing in something which you yourself do not understand is a risk. Do you know what IBM does, what their forecasts are, etc.?
• Blindly following another investor is not the best plan either. Mr. Buffett has several billion dollars and is a professional investor. I think we can safely assume that you do not have a billion dollars. Your investment strategies will need to be different from his.
• As pointed out in the comments, large investors inherently move the market when they buy, sell, or even talk about investing. This means that you will always pay more and sell for less if you follow behind Mr. Buffett.

Its not a bad idea to watch other investors especially those who can move markets but do your own research on an investment first. Your sole reason for investing should not be "Warren did it".

I think following the professional money managers is a strategy worth considering. The buys from your favorite investors can be taken as strong signals. But you should never buy any stock blindly just because someone else bought it. Be sure do your due diligence before the purchase. The most important question is not what they bought, but why they bought it and how much.

To add/comment on Freiheit's points:

• When a professional investor buys a stock, the risk/reward is calculated based on his portfolio. It may be optimal for his portfolio, but may not be for yours. Ask yourself: What does such purchase mean for my portfolio? Also, the opportunity cost can be different.
• It is not entirely true that you have to buy at a higher price than the professionals. One example: David Einhorn has recently purchased Resona (TYO:8308) at an average price of JPY547. Now (May 30, 2014) it is traded around JPY532. If you decide to buy Resona right now, you actually get a better deal than David Einhorn ;) Just for the record: David Einhorn is one of my favorite.

Update in 2020

Excerpt from Warren Buffett has sold IBM shares, and 'revalued' tech icon downward, cites 'big strong competitors' (May 2017):

"I don't value IBM the same way that I did 6 years ago when I started buying... I've revalued it somewhat downward," Buffett told CNBC. "When it got above $180 we actually sold a reasonable amount of stock."

[...]

"I think if you look back at what they were projecting and how they thought the business would develop I would say what they've run into is some pretty tough competitors," Buffett said. "IBM is a big strong company, but they've got big strong competitors too."

Berkshire Hathaway completely eliminated its stake in IBM in 2018. The investment gained less than 5% (including dividends) over about 7 years (source). In hindsight, the purchase of IBM stock was a mistake. Nowadays, the IBM purchase is used as a case study of Buffett's mistakes.

This is why you need to think for yourself, and not blindly copy others. You may be punished for not doing your own thinking. Gurus make mistakes too.

User Weiwei made an important point that Berkshire's risk/reward considerations and opportunity costs are likely to be different from yours. Keep in mind that Berkshire is a conglomerate, and the companies that it owns outright make up an increasingly large share of Berkshire's value relative to its stock portfolio. When you blindly copy Berkshire's stock purchases, you are also blindly using Berkshire's risk/reward considerations and opportunity costs that may be applicable to a conglomerate but not to you.