daily stats via email

[alexs]

I see that HR deletes stats after some days, i suggest to add a feature in the "raw access logs" in the control panel to insert an email address where logs are emailed every day before deleting them.

Stats are important for me and i can't loose them :-)

[troutster]

Hi,
I second that.
Scott
http://www.desert-storm.com

[ajc]

I third that !!

Allen

[foe]

I forth, i think it will be easier if you keep it on users account, but make them use thier own space for it. and allow them to delete it.

[troutster]

Yes, the space for the stats should be tied to my account. My stats were deleted without any warning. I would;ve liked to have been able to keep them, as it is important to me to know my site stats for the past.
Scott

[alexs]

As everybody know stats become bigger and bigger in a few days.
i think the problem is not where HR or users store them because they are large for HR and for users too (the server is the same), and users will have to delete them as HR does to avoid their space to be filled with the additional problem to pay attention to all their accounts to delete them before it's too late!
I think the daily email solution is the best it cleans the server from a large quantity of informations every day and let users to do their own stats with the software they want.

- Another consideration: Is there a way to launch the stat software (Analog or WebAnalizer) before deleting stats? In this way HR will have logs of only 1 day on the servers and stats always updated!! woww!

Give your opinion too...

[foe]

But in order to do their own stats they need logs, without logs there can be no stats.

[alexs]

They have logs and they should launch the stats programs before deleting them, some i know that webanalyzer can do this.
Anyway i'm shure HR knows how to do this and i hope they'll do this update soon :-)

[Brendan]

Hi,

We have several clients whos logs grow to several hundered megs over the course of a week. We have setup all new servers so that we dont have to clear the logs more than once every few weeks to a month.

-Brendan

[alexs]

I appreciate your interest in this issue but i think that when you delete logs (if you delete them completely), also once a month, there will be some logs lost.

I think the compressed raw logs by email (in off-peak hours) will be a good solution for all, eventually, to be combined with more space on the new servers reserved to logs.

Think about it...

[Brendan]

We're talking about several gigs of data, not a few megs of logs... it kinda limits what we can do via email.

-Brendan

[alexs]

I understand, but i think that users interested in making their own stats at home will download stats anyway and users not interested in stats will not subscribe to the daily email...
Anyway i think you'll choose the best solution for all!
Have a nice week-end!
Alex

[stratos]

Since statistics are very important for any web site, a good idea is to keep the most important statistic info (like no of visitors, country etc.) always on your server.
An email every week showing the number of visitors is not a bad idea either.

[4serendipity]

Here's a solution that I've just developed to capture my logs.

First I created a directory to hold my files ("logs" in the following example) and created a link to my log in this directory.

Next, I hacked together the following bit of PERL:

#!/usr/local/bin/perl
use Time::localtime;
use File::Copy;
$tm = localtime;
$a = $tm->year+1900;
$b = $tm->mon+1;
$c = $tm->mday;
$d = $tm->hour;
copy ("/home/username/logs/logfile", "/home/username/logs/$a-$b-$c-$d");

Finally, I used crontab to schedule this to run once a day.

This will generate copies of log named something like 2001-3-18-2 (for a copy of the log generated at 2am on March 3rd 2001).

[ajc]

4serendipity,

Thanks for the script. I think I'll start using something based on this to make sure I don't loose my logfile data.

Brendan,

How can I tell if I'm on a new server that will keep log file for up to a week ? When are log files deleted, do I need to run this script everyday, and at what time of day are they deleted ?

Allen

[4serendipity]

Here's an update on the script.
Instead of having to make a link, you could just point directly to your log.

So, you could change the last line from:

copy ("/home/username/logs/logfile", "/home/username/logs/$a-$b-$c-$d");

to

("/usr/local/apache/domlogs/your_domain", "/home/username/logs/$a-$b-$c-$d");

[ajc]

I've got the script running. But what time would you suggest is best to run it ?

Thanks,

Allen

[alexs]

The script is good but i think that if all users will run it together the server will be overloaded of datas.

I think that HR should think a solution for all users such as the one i suggested or something better ;-)

I suggest again 2 solutions:
1. users interested raw logs for stats will receive logs by email regularly by email, instead of putting on the servers self made scripts that will load it more than a central solution.
2. users not interested in raw logs will not receive the email and their raw logs will be deleted in a week after calculating webstats.

right?