First of all lets go over the idea that many companies have about running there eCommerce site on an in house server. For the most part this is just a bad idea unless you have some very specific requirements that make it mandatory to run it in house. In general most companies are just not equipped to supply the same level of maintenance and Internet connection as a dedicated hosting company can. Such as 24 hour staff and monitoring of servers, and redundant connections to multiple major backbone providers, not to mention the fact the hosting in general is not very expensive and the cost to host in house in equipment, and labor would be much more.
If you are planning on running a very large eCommerce site and you are in the market for a dedicated server then you definitely should. There are the obvious performance gains and better control with a dedicated server, but the top issue is security. With a dedicated server you do not have over people with access to your server.
If you are not in the market for a dedicated server then shared hosting is perfectly acceptable and extremely affordable. At this time even a really good hosting package will only run $19.99 per month. Just keep a close eye on your server performance and be prepared to try a different host if your server performance is not up to par.
There are the obvious things like tech support, reliability, basic features which I think are just common sense so will go over a few of the other things that may not be as obvious.
If you are running a professional eCommerce site then you are going to need to have access to the secure certificate on the server to encrypt communications for your order forms. If you are on a real budget then you may want to look for a company that offers a shared certificate on their server that you can use. But I feel that it is much more professional to have your own so if you want to purchase your own you need to make sure that the company you are using will be able to install it for you. The cost of certificates has come down drastically in the past few years so there is no reason not to purchase your own.
If you are on a shared server then one of the features that you should look into is getting a dedicated IP. The reason I say this is that I have hosting with companies that don't offer this and have run into problems because I was penalized for what others on my server were doing. For example if someone else on my server with the same IP was a spammer then everyone on the server can get flagged as spammer's. So when you attempt to send an email to a customer it may get refused because your entire servers IP address has been banned.
Again this is mainly if you are using a shared server because on a dedicated server you have full control and will automatically have access to your log files. But if you are on a shared server then you may want to check to see if this is available. The reason I say this is that I find it mandatory to run regular reports against my log files to see what is happening on the server. I think you would be surprised that as much as 75% of the traffic on your site may not even be coming from valid customers. It can be from bots, both search engine and other and it is the other that you should be concerned with. Also things like people spamming your log files with referrer spam. Attempts to hack your site and gain access to your server. And countless other things that I can not think of at the moment..