There was consternation on Wall Street. Non-performing assets -- virtual non-performing assets, for pity's sake -- were bringing home the bacon. The search engine sector had picked up $7 billion almost overnight. Was one of those distressing paradigm shifts in progress? Seriously, the question was posed.

Of course, that wasn't the issue. Investor confidence in Lycos was generated by strategic tie-ups with E-Loan,GetSmart, HomeShark, Realtor.com and CDNow which add up to a guaranteed marketing income of $30 million. Similarly, Infoseek and Excite went in for co-marketing deals with CNNfn and Software.net respectively.

The general perception is that search engines are in the business of indexing the Web, serving as its phone books. However, they are in the more general business of content aggregation. If that content comes from real estate brokers, so be it. Either way, engines remain the hottest properties on the Internet.

The point at debate now is whether they are likely to remain top Internet stocks. The chances are that today's top picks will be way down the scale in the very near future, unless they are able to diversify. The main problem is that search engines are compromised in their own core competency. They are supposed to lead users to the right places, not lead them astray.

Typically, people putting up new Websites send the addresses to a search engine. They are picked up by a spider, a small searchprogram that goes out on the Web, checks the addresses and follows all their links. It then brings his information back to the engine, which indexes it in its database, ready for use.

But to get halfway decent results, users of these databases have to know two flavours of Boolean (the mathematical and the verbal), know which engine prefers which, and how the keywords they are looking for are likely to be ranked. If you don't know the nitty-gritty, you're likely to feel a little lost.

Part of that problem has been solved by the meta-engines, or metacrawlers, named after one of the first such at www.metacrawler.com, which opened in July 1995 at the University of Washington. They take search queries and submit them to a range of engines. Mamma.com, for instance, queries seven of the standard search engines. The results, therefore, are far more likely to be useful than queries to a single engine. Metacrawlers, in general, are better content aggregators than standalone operations.

A second solution has comefrom specialisation. In the early days of the Internet, the sciences tended to group their servers together. After the Web was born, there were Webrings, nests of thematically related sites. And then there were specialised search engines which addressed only particular interests: medicine, astronomy, government, even alien sightings and gardening.

So today, there are two options. Either forget Boolean and search a metacrawler, or look up a specialised engine. The present hot-selling engines will have to learn to live with this competition.

But the engine sector in general will periodically show impressive investment gains like last week's, because it is technology-dependent, not human-dependent. Already, running costs are a mere 10 per cent of revenues. And as hardware prices continue to fall, profit margins will have to rise.