I initially wrote this in a PM, but I thought I could share this information with everyone, as some of you have no doubt been wondering what the hell is going on.

As you all know... Our hosting company is shit. When we moved with them, they were fine, but were soon bought out, and support was replaced with outsourced Indians (surprise, surprise). So now everytime I go in their IRC support channel to complain, I have the hardest time making them understand what I want. Not to mention some staggering incompetence...
We stayed with them because, for a while, they were the cheapest we could expect. See, it appears that their bandwidth counter was broken, because we were never charged and never had the "bandwidth limit exceeded" error for a while. But then we were plagued with the constant MySQL errors, until they upgraded their servers and migrated all their clients to the new, stronger servers.
...So then, we stopped having the MySQL max_connection error, but the bandwidth counter started to work again... oops! Well, we didn't mind paying for more bandwidth for a superfast site, but then we had other problems, namely the notification emails not reaching their recipients (and even having the mail service completely down at some point), our FTP access was often down for no reason, and so on... a real nightmare. It seems they can hardly do anything right.
And when the bandwidth burst again later last month, we decided to wait it out, and when it got reset... all the old pre-migration problems returned. I have yet to figure out why this happened.

We did receive several emails from various offers, most of which have NO idea just how big we are (like "look we'll offer you 100 gigs of bandwidth per month that's more than enough rite??"). But one guy, who works in a hosting company close to us in Montreal, is a fan of our site and has shown willingness to help. He initially offered us free hosting on his server, but he also underestimated our traffic. Still, he and HellBlazer talked for a while on MSN and he gave us tips to optimize our scripts and setup so that the site would be less resource-intensive. Still, even he doesn't seem to know why the site was running so smoothly in March and then suddenly came crashing down at the immediate turn of the month, but he's been of more help in that one MSN session than our host support has ever been, so that tells you something about their service. So he's looking into his side to see if he can come up with some paying solution.

Also our host now recently offered to upgrade us again, to a VPS (virtual private server), which would be (according to them, and others too) more suited to our needs, but of course costlier... so, HB and I are finally looking into that donations thing now. Still, he offered a discount on their VPS plan just for us, so it's not all that bad. It'd be around $35 more per month than what we are paying right now.

We genuinely and gratefully appreciate every offer of donations we received. But keep in mind that moving and upgrading a site like this one is a bitch, especially with such a large database and so many image files. Some of you may remember the last migration in December, which paralyzed the site for a few days (we were promised a smooth, near-seemless transition and something to be finished within a few hours, not days ). Every single one of our upgrade or migrations with our host so far has been catastrophic, so if we do upgrade, we're reluctant to keep going with them (I'm fed up with the shitty service myself). Other solutions are usually costlier, but we might get better service. Or not, they could be worse... so you see why we're reluctant to move. Imagine if we lose the site again for a few days, only to end up having the same problems? We prefer to be sure of what we do before we commit.

Anyway, what I'm trying to say is: yeah, we'd love donations, since we're already paying over $100 USD a month out of our own pockets. But we understand that you would all be very frustrated to donate, and not see immediate improvements. So I ask you to keep all of this in mind if you do decide to donate, as we might set up a Paypal donation button before we start moving (if only to appease the "please accept donations!" suggestions we keep getting ). Still, crap server or great server, it all costs us a lot of money regardless, so we'd still appreciate it. But it's up to you, of course.

Thanks to everyone for your patience. If you think this whole thing is irritating, understand that we are too, tenfold.


Edit: I'll add that we received a few offers of having free hosting in exchange for banner ads and the likes, and suggestions of having paying membership accounts and the likes. Trust me, the answer to that is a resounding NO.

People say bad things about Paypal... don't know how true those rumors are, just makin' sure we won't need to deal with any further trouble.

I thoroughly applaud your consistency of principles (no, that's not sarcasm). It's a tough jam, but I'm not particularly worried after reading that.

Nice post.

...?

I've used paypal so many times before with eBay, services, etc ... its really easy and simple. I'm just not sure what these "rumors" are?

Anyways, its good to have a little more information on the issue here.

If you want to stretch the bandwidth a bit more you could losslessly compress the images (band logo + image and album covers) with jpegoptim. Even if you got a conservative 5% off each image that could really add up.

Seconded

Aforementioned rumors.

Interesting. Though in -this- case I don't see how a paypal route would hurt when we know we're sending money to MA. And I doupt many of us here would send a large enough amount anyways to the point where we may need it back or whatnot.

i have paypal, i love having paypal, its an incredible bitch ill admit, but its prettymuch free
i dont really have money to give
but i can send you 10 canada dollers
i may be tottaly unkown on this site
but ive been using it forever.. just reluctant to post any bands.. well i tried and failed once...
ANYWAYS

get paypal, i giv u col har interweb cash.. kay?

You should get hosting from these folks. There slogan is simply "We're good", and I've found no reason to doubt them.

www.positive-internet.com

Thanks for clearing a few things out

Is it possible, that you'll write here (or PM it) the approximate disk
usage/bandwidth/mySQL usage for Encyclopaedia Metallum?
I work in a hosting company (we'r connected to Verio/Rapidsite) and I'd
like to compare the prices (I understand you pay over 100$ monthly), maybe
it will be worthwhile.

I've used it too, but only to make payments, never to receive.

Of course my internet connection picks today to suck. It was fine all those days when you guys where having hosting problems and I couldn't get through to the site, but now....

I salute Morrigan and Hellblazer for running the site on their own expenditure for so long. Quite a funny post, especially the part about the Indians . The scum among us who couldn't muster a good grade or those who want to make a quick buck before they actually join college (usually the former) work get into call centres. These people too are like trained sheep, they speak with a pseudo-American accent, use Americanized versions of their Indian names and they're hardly trained at what they do! Usually, they just have a big manual which they sift through to verify the complainants problems. Of course, many call centres get such stupid queries you don't even need to be qualified to answer them. Can't blame them though, after all its your dimwit CEO's who actually give them the jobs and as long as it bolsters the Indian economy, I have no problem with it .

Thanks for the explanation of what's going on. After all the music this site has helped me discover the least I could do is toss a couple bucks in your direction.

Of course .. and everybody wants to outsource because it's so much cheaper, and these companies are all about dollars in the coffers and not anything remotely connected with customer/client satisfaction. I still think it's a self-defeating proposition because, after all, the customer is the man who pays, but I'm not an economist, thankfully.

So 100 GB a month of bandwidth is not nearly enough? How much traffic are we talking about here? BTW, it's no exaggeration to say that MA has grown quite huge ... I mentioned in the IRC channel a while ago that those of us who post here on the fora or write reviews and so on are the barest percentage of those who hit the site to look up bands and such, and you can regularly here people from complete newbies to "experienced" metalheads at shows here talking about using the site to find information, etc.

Dude, 100 gigs would be gone in three days. We use between 30 and 35 gigs per day so... near a terabyte per month.



Rounded up... 10 gigs in files, 1 gig for MySQL. With the database of course being hammered constantly. Apparently the search was causing huge loads on the DB, and as you can see now it runs great without it, but... last month it was fine even with the search so, complete mystery.

paying membership accounts would be a crappy idea, donations is fine.
i wonder why you never asked for them before.

anyway, good luck with the site.

I am sure lots of hosting recommendations are coming in .. and I understand a large site migration is a bitch (I do enough of them in my day job) but have you looked at Dreamhost?
http://dreamhost.com/hosting.html

I use them to host my site, admittedly my site would only be a drop in the bucket compared to what MA does, and I am quite happy with them

Their bandwidth per month is very high. For example the Level 4 plan give you over 4 TB per month and it increases by 40Gb per week.

Regarding a migration .. there shouldn't really be any downtime. Lock down anything that would update the database. Start migrating the everything (while the 'old' site is running). Test the hell out of things on the new hosting place. When things look good switch the DNS.

Feel free to message me privately if you need any help with a game plan on migration. As I said, it's what I do.

Thanks for letting us know what's going on. I'll tell you, though...if a paypal donation starts up, I'll be happy to donate.

I don't have Paypal, but I would get an account solely for making donations ... I would be a huge leech if I didn't donate considering how much I use this site.

I think this is the best option.

I'm pretty poor, but I'd donate what I could.

I'd like to clarify that bandwidth is not the sole (nor the most worrying) issue. CPU usage and the strain the heavy MySQL usage puts on the server is a more significant issue. Even cheap shared hosting who are generous in bandwidth could not accomodate us, which is why our host offered to put us on a VPS. :\

You should take your time that is necessary to examine where the failure in the system is and how to fix it properly. From a distant point of view the errors with MySQL sound rather strange, especially when the error occurs, perhaps an obscure outcome of the MOPB, but hell knows. Perhaps this also lead to the unexplainable behaviour of the search-engine.

I did not expect that such a large bandwidth is used and so much data flows over the net, just because of MA; damn you point-whores!

I do not use paypal so far, but perhaps you can offer an additional way of sending the money.

what a load of tripe! yes... it's true a job in a call center isnt what one would consider a particularly wonderful career choice... but to go ahead and single out indians for this is a load of shite ... ! customer care sucks all over the board ... not jst india... even call centers centered in america n europe are quite despicable in terms of service .... !
'nyways... good to see the site is up and running again!
and i think it'll be great if u open the site up for donations ... i'm sure a lot of us around would love to help out financially whenever we can ...!

Iron Will Radio will be glad to donate.

I have to deal with call centers on a regular basis. This is not a racist comment but when the person on the other end barely speaks english it's a problem .. a BIG problem. If your job is to verbally communicate with english speakers then do something to reduce your accent!

I think the majority of problems people have are not due to a lack of education by call center staff. They stem from a breakdown in communication.

I use this site dozens of times a day, so I did a bit of shopping around as far as dedicated hosting goes. If you guys are putting huge load on the CPU a dedicated server is the way to go, as even a VPS is still shared hosting. I found multiple places selling plans that seem to be within the budget, especially if you start accepting donations. For the following I'm using $150/month as the absolute cutoff, seeing as that is probably more than you are already paying. That also seems to be the price point at which you could get everything you need, and then some.

Here's what I found:
The Planet offers dedicated servers starting at $69/month. At around $150/month you can get a very decent dualcore xeon server with 250gb of storage, a gig of ram, and 1.5tb of bandwidth/month.

ServerPronto has dedicated plans starting at $30/month, but doesn't seem to offer the same bang for the buck as the previous site. Once you approach the $150/month mark, it's only 800gb bandwidth, 160gb storage, and an AMD Athlon 3000+.

$150/month at Server Beach can get you 160gb storage, 1 gig of ram, an Athlon 3500+ and 2tb of bandwidth. Lesser plans start around $75/month.

Layered Tech servers at $140/month can get you a server with a Core 2 Duo E6600, a gig of ram, 160gb storage, 1.5tb bandwidth.

While some details were given in this thread, I'm not sure of the exact needs across the board, so much of this is guesswork as to whether or not it would be adequate. I would definitely check with any future provider in advance should you choose to move.

I'm sure there are more out there, but that is what I found in a quick search. I also want to make it clear that I have no personal experience with any of the above, so I cannot vouch for them. I wish the M-A crew the best of luck in dealing with the site, and I dearly hope that everything works out favorably.

Also, I am not an expert! I am a hobbyist at best, so if anyone with more knowledge of the subject would like to offer their advice, you should probably take it over mine.

Cheers!

About the bandwidth, if I remember it correctly, back in 2004 we were already using more than 100G.

Have you checked out Dreamhost ever? They offer 160gb of space, 1.6TB of bandwidth, and supposedly very good MySQL/PHP/customer support for something like $8/month. I know many people who use it and love it, but I don't know if that's enough space or bandwidth for you since I don't really know your specifics. Anyways, dreamhost.com if you're interested... I know a few coupon codes to get you half off as well.

EDIT: They have plans with double that bandwidth and more and more space:

http://www.dreamhost.com/hosting.html

christkiller alone uses 10gigs a day.

Touche'

I said us in the post. I'm an Indian myself, do you think I'd go around criticising my own people unnecessarily? Unfortunately whatever I said was true. And yeah, I agree completely with customer service being bad. Very little can be accomplished by speaking over the phone with another person who has no idea what circumstances you're in, right? What I'm worried about though is that the 'call centre stereotype' will be carried over to all the people in India and abroad, when it is only the less (or perhaps, least) educated among us who end up there.



People in call centres have a decent amount of knowledge in English, I believe their only problem is the difference in accent. In fact, they all speak in English instead of their own regional languages to be 'metropolitan' .

But he uses them well.

I could NOT agree more. Do it MA> I'd be more than happy to throw a couple of quid in the pot. This site is exactly what it says on the tin - my metal encyclopedia, without which life is not the same.

What's the mailadress you guys are using for Paypaldonations?

Vibrations Of Doom Magazine uses this site RELIGIOUSLY as a resource, ESPECIALLY for 80's metal! To many of you reading this, let me say that if I EVER need ANY info on this really rare and obscure 80's metal gem (say, I need an album cover or label info), I ALWAYS go to Encyclopedia Metallum FIRST. Then if the covers aren't there I go to ebay! I have always provided scanned covers for those 80's metal albums E.M. didn't have, so needless to say I'm a HUGE contributor... Donations? I'd be more than happy to stand on the front line and throw some cash their way... My site kinda works on the same principle, though of course most of my diskspace useage comes from the hours upon hours of streaming audio you can listen to... (that includes the weekly 2 hour radio show, the over 1,000 rare, classic and out of print 80's metal albums you can listen to IN THEIR ENTIRETY, and the soundfiles for TWO issues' worth of CD reviews). Bandwidth concerns I can understand, however I am seeing plans become MUCH cheaper recently and offering WAY more than I ever had with ANY of my hosting plans...

Definitely, the guy is a beast.