What would cause cPanel to show 5TB Bandwidth but a hosting company to report 10TB?

[contract]

Dealing with something that is confusing the hell of out me for months now.

cPanel is reporting (example) 5TB TOTAL bandwidth for http, email, ftp, etc.

Host is reporting 10TB total bandwidth usage.

Awstats reports 150GB/day, web host provider reports 400+GB/day and growing..

....

Back story:

I switched servers. Same theme, same site, same everything, just a better server.

Bandwidth jumped from 5TB/month to 15TB/month.

Traffic is growing, but there was not a major spike that would cause bandwidth to jump 3x times in size in a weeks time...

....

- I haven't downloaded anything myself.

- I pay for a seperate backup service from the hosting provider... So that band shouldn't be in there.

- Site is only 25GB, nothing huge.

- Cut my server file size for media by roughly 40% and cut the homepage/page themes down 1.5MB 50% per load. Now, there should be some serious savings there... When you consider the before and after it should be dramatically lower...

 

[CCarter]

It sounds like backend processing is communicating with something that's being disregarded by some of your reports.

I also recall once you said something about using a compression plugin for images - that might account for the jump.

Awstats reads the apache2/nginx server logs - doesn't necessarily mean that it will read everything, like example a backup to an offsite location - rsync or something, that movement of the data wouldn't hit apache/nginx's logs, but it would hit the syslog.

Also, something to consider, if you have backups stored on the server, and they are aggregating, without being excluded in future backups, you essentially are backing up old backups in the newest backups. That's a possibility.

Spam might be a problem too - if you are getting a flood of spam, but gets immediately deleted it might not hit some of the higher level logs you are used to monitoring.

A 10TB discrepancy is too huge to overlook. You really need a server administrator to come and login to your server and figure out what's going on. I always recommend @SmokeTree - dude gets the job done and is fairly priced.

 

[SmokeTree]

This is kind of long because I am including some info that might also be helpful to some of the builders here that are trying to get a better idea of what's eating their bandwidth. Sometimes a ton of money can be saved with a couple of well placed firewall rules after using some of the tools mentioned below.

I agree with @CCarter here (BTW, thanks for the props ) . Your Hosting provider is measuring the bandwidth coming across the network (virtual) Interfaces, which works pretty much the same way a water meter or electric meter works. It's the only way to measure the total network bandwidth. This is the first place to start.

Knowing where your traffic is coming from or going to is going to tell a big part of the story you're not seeing now. In cases where you have root access and SSH, what I normally do is put something like ntop on the server. Here's a link to the next generation version, called ntopng. http://www.ntop.org/products/traffic-analysis/ntop/ This one will give you a breakdown of traffic by IP, interface, etc. You can get a real good idea of where your traffic is coming or going to. Your cpanel might have other options/packages for bandwidth monitoring. I only know Linux/Unix from the command line and don't know much about various cpanels.

Next is to take a look at processes on the server. If there is a process on your server that is causing the bandwidth issue, sometimes you can catch these in the act on the command line with htop (http://hisham.hm/htop/). Other times you'll need to dive deeper. Running "ps aux" (might need to do "ps -aux") can give you a good start of what's running at that point and time. If you want to see pretty much everything happening on your system in real time, you can install this: https://my-netdata.io/

I'm not sure what kind of limitations you have on your setup, but in almost all cases, you're going to want SSH access with root or a "sudo" user that can execute commands as root. Without it, a cpanel is a lot like a car with a lock on the hood.

I hope some of the things I mentioned above help you and/or other builders to sort bandwidth related issues out. If you run into some snags, I'd be glad to help.