downloads.mdb

Introduction

Statistics in the internet are very difficult. At best you can do intelligent estimates and read out tendencies. It is almost impossible to get absolute numbers from server logfiles.

»But downloads are different. All you have to do is count them. Right?«

You can of course count the clicks onto a downloadlink or a button. But all you get this way are clicks, not downloads. Who can say whether a visitor waited to the end of a larger file or cancelled the download?

»Well, then we will check the logfile, which contains the files downloaded.«

Correct, but what if somebody use a download manager? Such a program will split the download into smaller bites, download these simultaneously, if possible, and thus make downloads faster. But each bite gets a separate entry in the server log. Must we turn on our pocket calculators now? Of course not.

Database

Downloads.mdb by blechtrottel brodaktschns tackles this very problem. It is an Access database which counts the downloads in a server log. It counts downloads for a certain file per day and ip address. Thus, it can count downloads made with download managers, which download files in smaller pieces, as it relies on the sum total for an ip address rather than single entries.

Version History

• Version 2.1 (January 2010)
  Easier (more user-friendly) start with the database
  Several Access error-messages caught
• Version 2.0 (May 2009)
  Your logfile may have different file extensions (txt, csv, tab, asc, tmp)
  Select-button opens last folder
  File paths can now be typed into the text fields
  Downloads list (was downloads.txt) can have any name
  All download files are listed (even those with 0 downloads)
  new language Irish (Gaeilge)
  smaller file
  minor bugfixes
• Version 1.1 (January 2007)
  speaks German and English

Limits

• Should a download stretch over midnight into a new day, it will not be detected correctly, it will be counted as two incomplete downloads on two different days and be dropped from the statistics.
• If a user starts a download e.g. on a laptop in a network (office, WLAN) and continues this download in another network (at home, in another WLAN), this download cannot be detected, because usually the ip address will change as well.
• The same holds true for interrupted downloads on a computer without a constant internet connection. Here too, the pc will get a new ip address when connecting.
• On the other hand it may happen that e.g. in an internet café several persons consecutively start downloading a file on the same computer but do not finish it. The total may count as one or more downloads, for the server logs the computer only, it knows nothing of the persons using it.

Preliminary Remarks

To work with downloads.mdb you need access to your serverlog. If you are not sure, ask your provider.

Our database works with logfiles in Common Logfile Format (CLF) only and was tested in a German Access only. Should it not work with your CLF logs, please send us about ten lines from your log, so we can adapt it.

Access only allows a limited number of file extensions for the logs. Unfortunately log is not among them. If you select a file with an extension that is not supported, downloads.mdb will automatically rename the file to txt, to be able to work with it. If you should not like to keep it, change the extension back again once you have done your statistics. Downloads.mdb will not make any changes to the content of your logfiles!

Usage

1. You can test the database with the testransferlog.txt and testdownloads.txt provided. If you want to work with your own log the create a downloads list, a file with the extension txt in a folder of your choice. In this file add a line for all the downloads you want to count:

   name;from/root/filename;bytes

   So, for www.mywebsite.com/download/ie4.zip include:

   IE4;download/ie4.zip;789270

2. At the moment, the database supports German, English and Irish (Gaeilge). You can select one of them in a dropdown field named Sprache/Language. If you want to translate the database into another language, please send us a word list. The terms used can be found in a table in downloads.mdb called Texte.
3. Click the button Select next to the field Downloads and select the downloads list you created. Now select the logfile for your website by pressing the second button labelled Select and after a short while you should see a result similar to our screenshot. Alternatively, you can type the file paths into the text fields.
4. You will need the Calculate button only if the data in your files has changed while their names have not. Whenever you select another file the calculation will be toggled automatically.
5. If a file shows not even 0 downloads then it was not found in the logfile. This either means that nobody have hit the file or may indicate that there is something wrong in the corresponding line in the downloads list (wrong path, typo in the file name etc).

Download

You can download the database as downloadsmdb.zip. It contains a testdownloads.txt and a testtransfer.txt, too.

Final Remarks

• Our database has successfully been counting the downloads for our projects for about three year now. These projects are all hosted on Apache servers, we could therefore not test it with other servers' logs. Please send feedback.
• The database has not been locked or protected. If you know your way in Access, you can adapt it to your personal needs. Comments are welcome anytime.
• The statistics you can create with downloads.mdb (and similar programs and services) are about as acurate as the circulation figures for a newspaper. No one can say how many readers a newspaper really has, several people may share one copy while some other copies end up in the bin unread. And even if we could count downloads exactly, we still would not know if the files are actually used ...