EBTST - EBT stats tool

Various EBT statistics can be posted here

Moderators: Phaseolus, Fons

Post Reply
User avatar
dserrano5
Euro-Master
Euro-Master
Posts: 3773
Joined: Sun Jan 31, 2010 11:59 am
Location: Madrid, ES

EBTST - EBT stats tool

Post by dserrano5 »

Hi,

I've been working on a statistics tool for us EBTers for the last months, given that NIG's lack of updates have decreased its usefulness. I completely fail at naming things so I just stuck with "EBT stats tool" or EBTST for short. Now that it has reached a point where its feature set is more or less complete and all the critical bugs hopefully have gone away, it's time to release it to the world.

Usage

You don't have to download or install anything, EBTST lives here and it's usable right now. Registration is needed, though, otherwise the program wouldn't be able to keep track of each users' data.

Upon registration, you are invited to upload your CSV files. Depending on their size and the speed of your connection, that might take a while. Hopefully there's a progress bar to inform you about it. When the upload ends, EBTST analyzes the CSVs searching for errors in the notes and figuring out which ECB president signed them, among other things. This "import" procedure has its own progress bar too. After that, you're directed to the "Information" section, which is pretty much the same as NIG's. From that point on, you can browse the sections at the left.

This is the "private" part of EBTST, only you can see it. For generating the public stats, you can use the "BBCode/HTML" link that is near the bottom of the sections listing. Check the boxes of the sections you want to publish and then click that link. When the process ends, a new window/tab will open containing the relevant BBCode (which you can copy/paste to the forum) and a link to the HTML stats, which will be of the form "https://dserrano5.es/ebt/ebtst/stats/<your username>".

Beware, though, that any pop-up blocker that you may have installed will probably prevent EBTST from opening that window/tab. If that's the case, you'll have to allow pop-ups from dserrano5.es. Rest assured that I don't have plans to insert any kind of advertisement in the program ;). Related to this is the need of having javascript enabled in order to the progress bars to work, so any javascript blocker you may have installed (e.g. NoScript) has to be configured to allow javascript from dserrano5.es.

Highlights

EBTST is open source 8). There's a github repository with the source code, which you can download and use on your own if you like.

EBTST is fully translatable. Right now there's a file called es.txt containing the Spanish translation. If you want to translate it to some other language, download it (this is the raw copy) and change the sentences at the right of the equals signs. Most of them are self-explanatory; however the help related paragraphs are different. Check out the file en.txt for their meanings. When you've finished, send me the edited file and I'll include it in the application.

Lowlights

Several sections feature a couple of Unicode characters that may not show correctly in your browser (some test users reported that they were unable to see them). If there's a significant fraction of users with problems, I might change them. They are:
  • U+2714 (HEAVY CHECK MARK, ✔)
  • U+00D7 (MULTIPLICATION SIGN, ×)
  • U+2192 (RIGHTWARDS ARROW, →)
  • U+2194 (LEFT RIGHT ARROW, ↔)
Notes which have been found more than twice (triple hits and beyond) have been poorly tested. Things might break regarding those.

Users with more than 5000 hits may notice a delay at the end of the "import" procedure (you know, the progress bar reaches 95% and then seems to stop there ;)). I will fix this but in the meanwhile, please be patient. A minute or two should be enough to let the process end.

The login and register forms don't have any kind of error reporting ("bad user/password", "invalid user", "already existing user" and so on). Needless to say I will correct this too.

EBTST could be faster but it's installed on a machine with no more than 4 Gb RAM so I had to sacrifice some speed to keep the memory usage low. Power users will see that their operations are somewhat slow, but this is all I can do. I tested with 1.9 millons of notes.

Acknowledgements

I want to thank the spanish community for trying EBTST and uncovering lots of errors and problems, ranging from trivial to fatal, as well as suggesting new ideas. They are, in chronological order, Celtium, Adela Merchan, Vicente1712, Skybang, Angelino, minyonet, rakoon, TXATXE, Manaies, dargqui, lordsox, golosinas, quetzal02 (now living in France), Akai-DDragonNk, Victorius, gugus, Murbella, bendder, Crawler360 and groucho. Some non-spanish users who also helped were :flag-it: giulcenc&family, :flag-nl: MDeen and :flag-de: Math Murderer. Thank you all!
User avatar
Kleman
Forum Moderator
Forum Moderator
Posts: 16062
Joined: Mon Nov 14, 2005 12:24 am
Location: Bordeaux
Contact:

Re: EBTST - EBT stats tool

Post by Kleman »

Wonderful !!
congratulations !! :D
Mes stats

Monde: 34
France :flag-fr: : 1
Local (Bordeaux): 1

Notes: 882 825
Hits: 1436
User avatar
Ewri
Euro-Master
Euro-Master
Posts: 4597
Joined: Mon Jan 07, 2008 10:40 pm
Location: Malta

Re: EBTST - EBT stats tool

Post by Ewri »

Thanks to all, especially dserrano for this.

I will update my details soon.
User avatar
dserrano5
Euro-Master
Euro-Master
Posts: 3773
Joined: Sun Jan 31, 2010 11:59 am
Location: Madrid, ES

Re: EBTST - EBT stats tool

Post by dserrano5 »

A couple of things I forgot:

Usernames and passwords can contain spaces and Unicode characters. The only forbidden characters are:
  • &
  • <
  • >
  • "
  • '
I tested the application with Firefox, Chrome and Opera on Linux. I guess other browsers should work (at least one test user uses a Mac, and most use Windows), although I wouldn't be surprised if Internet Explorer didn't, due to some things I'm doing with the javascript. I don't plan to support IE.
User avatar
gabonik
Euro-Master
Euro-Master
Posts: 1759
Joined: Sat Dec 20, 2008 6:15 pm
Location: Ravenna - Fornace Zarattini - Italia

Re: EBTST - EBT stats tool

Post by gabonik »

Compliments, very beautiful, I find very useful the part: Bad notebook. An explanation, if possible, in the list of my banknotes further to list the banknotes to signature Trichet - Duisenberg - Dragons there are also 17 unknown. From thing you/he/she can depend? Thanks and again compliments.

:)
User avatar
Kleman
Forum Moderator
Forum Moderator
Posts: 16062
Joined: Mon Nov 14, 2005 12:24 am
Location: Bordeaux
Contact:

Re: EBTST - EBT stats tool

Post by Kleman »

I've shared your fantastic tool in french forum:

http://forum.eurobilltracker.com/viewto ... 34&t=50051" onclick="window.open(this.href);return false;
Mes stats

Monde: 34
France :flag-fr: : 1
Local (Bordeaux): 1

Notes: 882 825
Hits: 1436
User avatar
yli
Euro-Master
Euro-Master
Posts: 5870
Joined: Mon Dec 20, 2004 4:34 am
Location: TILBURG
Contact:

Re: EBTST - EBT stats tool

Post by yli »

dserrano5 wrote:...although I wouldn't be surprised if Internet Explorer didn't, due to some things I'm doing with the javascript. I don't plan to support IE.
With IE the upload doesn't work..... but I am now testing it with Chrome, and something is happening :wink:

Some minor problems :

missing flags for Image Bosnia Herzegovina and Image Serbia

In the travelstats same locations in other countries are together. For example the notes of :flag-be: Comines and calculated with :flag-fr: Comines.

Numbers in a row :

a note with 6 numbers in a row is also a note with 5 numbers in a row.

EBTST :
57 note(s) (0.07% of total) have 5 numbers in a row.
3 note(s) (0.00% of total) have 6 numbers in a row
NIG :

60 notes (0,1 % of all notes) had at least 5 same numbers in a row.
3 notes (0,0 % of all notes) had at least 6 same numbers in a row.

hitlist : notes between hits isn't correct. I haven't entered notes since my last hits, but it's saying 71 notes between for the last four hits.

But for the rest it works good, and looks really nice :D
Last edited by yli on Sun Sep 16, 2012 7:06 pm, edited 1 time in total.
Jelle zonder internet. Het was een leuke hobby, maar niet mijn hobby. Een leven zonder internet is mijn ding. Een leven van geld uitgeven in plaats van tracken zeg maar.... hahaha. Ja, er mag gelachen worden.
Inloggen op site lukt niet meer, maakt niet uit. Ik snap toch niets van nieuwe biljetten :lol:
User avatar
an-148
Euro-Master
Euro-Master
Posts: 4596
Joined: Wed Oct 11, 2006 5:04 pm
Location: summer: Ivanica, BiH ; winter: La Calamine (Liège) BE
Contact:

Re: EBTST - EBT stats tool

Post by an-148 »

also missing: flag of Serbia & Montenegro before independence (and separation of bills registered in common country before independence)
:wink: :wink:
http://meine.flugstatistik.de/image/an148.gif" onclick="window.open(this.href);return false;
User avatar
jonman
Euro-Master in Training
Euro-Master in Training
Posts: 819
Joined: Sun Mar 06, 2011 10:58 pm

Re: EBTST - EBT stats tool

Post by jonman »

first of all: this is great, dserrano!!! :D :D
very good!
just a few comments at which you can look at whenever you feel like it or when you want to make minor improvements to your program :wink:

another interesting stat i would be very happy to be informed about is series of uncirculated notes i have entered. so maybe your program could scan the csv file for notes with consecutive serial numbers and list the series. it would be very interesting to see at which point in time you were able to get a certain printer code/serial number combination in an uncirculated condition. coming to think of it, you didn’t implement the “friends” notes stat from nig showing notes that must have been close together in an uncirculated bundle but which have come through your hands at different times or even at different places. is there a specific reason why you left out on this one?

for some reason the graph of the statistic on the historic development of your hit ratio (see hit summary page) seems to be slightly off at some points. have a look at my screenshot:
screenshot.jpg
screenshot.jpg (39.3 KiB) Viewed 17448 times
as shown above my worst hit ratio ever was at 1:129 with the graph showing 1:138. and my best hit ratio ever would be 1:54 whereas it looks rather like 1:59 to me. (the current hit ratio looks alright to me :?:) just on a side note: i have no idea how the graphs are being created or if there is any chance of changing it, but if there was i would very much appreciate a change in the looks. i noticed, especially on the travel statistics page in the diagram about the cities with most notes entered, that the lines are very thin, and, to be honest (but really, do consider this just as the tiniest possible way of how you could improve your program in my eyes) i liked the looks of the nig diagrams more :oops:

last but not least i was going to create a de.txt, but the en.txt you offered doesn’t seem to be as large as the es.txt (only showing the help section), and i’m not that good translating from spanish to any other language :wink:
now, all in all, thanks a lot for your great work there, it is very much appreciated!! :wink:
My profile
22.9.14 90. :note-200: entered
9.9.14 50. :note-500: entered
17.8.14 300. hit occured
12.8.14 550. :note-100: entered
28.7.14 5000. :note-10: entered
16.11.13 3 years of EBT
10.9.13 25000. note entered
30.7.13 15000. :note-5: entered
23.7.13 2500. :note-20: entered
7.2.13 1000. :note-50: entered
jrnsndrs
Euro-Master
Euro-Master
Posts: 2118
Joined: Wed Feb 13, 2008 8:27 pm
Location: Tilburg
Contact:

Re: EBTST - EBT stats tool

Post by jrnsndrs »

After 10% uploading, it stops with an Internal Server Error.

In the meanwhile, I have started the Dutch translation. I will publish it in the Dutch forum for review.
Last edited by jrnsndrs on Sun Sep 16, 2012 7:31 pm, edited 1 time in total.
"of all wars, religious wars are the most detestable, since they are waged for no tangible gain, but only to impose a set of arbitrary credos on another."
--- Unspiek baron Bodissey.

-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
User avatar
jonman
Euro-Master in Training
Euro-Master in Training
Posts: 819
Joined: Sun Mar 06, 2011 10:58 pm

Re: EBTST - EBT stats tool

Post by jonman »

jonman wrote: last but not least i was going to create a de.txt, but the en.txt you offered doesn’t seem to be as large as the es.txt (only showing the help section), and i’m not that good translating from spanish to any other language :wink:
:x
never mind this one, i got it now :roll:
My profile
22.9.14 90. :note-200: entered
9.9.14 50. :note-500: entered
17.8.14 300. hit occured
12.8.14 550. :note-100: entered
28.7.14 5000. :note-10: entered
16.11.13 3 years of EBT
10.9.13 25000. note entered
30.7.13 15000. :note-5: entered
23.7.13 2500. :note-20: entered
7.2.13 1000. :note-50: entered
jrnsndrs
Euro-Master
Euro-Master
Posts: 2118
Joined: Wed Feb 13, 2008 8:27 pm
Location: Tilburg
Contact:

Re: EBTST - EBT stats tool

Post by jrnsndrs »

jonman wrote:
jonman wrote: last but not least i was going to create a de.txt, but the en.txt you offered doesn’t seem to be as large as the es.txt (only showing the help section), and i’m not that good translating from spanish to any other language :wink:
:x
never mind this one, i got it now :roll:

You can translate from the raw code. That one has also English. (link in the first message)
"of all wars, religious wars are the most detestable, since they are waged for no tangible gain, but only to impose a set of arbitrary credos on another."
--- Unspiek baron Bodissey.

-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
jrnsndrs
Euro-Master
Euro-Master
Posts: 2118
Joined: Wed Feb 13, 2008 8:27 pm
Location: Tilburg
Contact:

Re: EBTST - EBT stats tool

Post by jrnsndrs »

jrnsndrs wrote:After 10% uploading, it stops with an Internal Server Error.

(...)
Can this be caused by sending unzipped files?
Wait a moment, I will try.
That was indeed the problem.
"of all wars, religious wars are the most detestable, since they are waged for no tangible gain, but only to impose a set of arbitrary credos on another."
--- Unspiek baron Bodissey.

-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
User avatar
dserrano5
Euro-Master
Euro-Master
Posts: 3773
Joined: Sun Jan 31, 2010 11:59 am
Location: Madrid, ES

Re: EBTST - EBT stats tool

Post by dserrano5 »

gabonik wrote:Compliments, very beautiful, I find very useful the part: Bad notebook. An explanation, if possible, in the list of my banknotes further to list the banknotes to signature Trichet - Duisenberg - Dragons there are also 17 unknown.
Regarding the unknown signatures, they are most probably caused by bad notes, ie notes with errors. Look into your "Bad notes" section and try to fix as many as possible. Then the notes with unknown signature should dissapear.

If anyone encounters a situation where he/she has notes with unknown signature and there's no "Bad notes" section visible (ie, no bad notes in the database, congrats!), that's a bug in ebtst and I'd love to hear about it.

yli wrote:missing flags
Initially I used the standard ISO-3166 for the country codes and flags, but during the testing I found some notes from Kosovo, that doesn't exist in the ISO. Well, I guess Kosovo isn't the only one :). Thanks for reporting.

yli wrote:In the travelstats same locations in other countries are together. For example the notes of :flag-be: Comines and calculated with :flag-fr: Comines.
Wow, I'll carefully look into this :).

yli wrote:a note with 6 numbers in a row is also a note with 5 numbers in a row.
But that's not interesting to the reader. I'm intentionally counting notes with 6 and only 6 numbers in a row.

yli wrote:hitlist : notes between hits isn't correct. I haven't entered notes since my last hits, but it's saying 71 notes between for the last four hits.
Thanks for this. Those statistics regarding hits are difficult to compute and, although it's a well tested part of the application, I'm not surprised to see small errors like this.

an-148 wrote:also missing: flag of Serbia & Montenegro before independence
Argh, why do countries keep splitting and joining again? Couldn't we all be a big fat worldwide nation? :)

jonman wrote:you didn’t implement the “friends” notes stat from nig […] is there a specific reason why you left out on this one?
Lack of interest, basically. But you're not the first one to tell me, so it's already on the TODO list.

jonman wrote:as shown above my worst hit ratio ever was at 1:129 with the graph showing 1:138. and my best hit ratio ever would be 1:54 whereas it looks rather like 1:59 to me.
Interesting, other users don't have this problem :). I'll have to investigate.

jrnsndrs wrote:
jrnsndrs wrote:After 10% uploading, it stops with an Internal Server Error.
Can this be caused by sending unzipped files?
No. Your CSVs have a special character at their very beginning (called the Unicode Byte Order Mark, or BOM) and I wasn't expecting it. I have already changed the code to support it (you weren't the only user to be stung by this).

Regarding compressed files, in theory there's support for them but some test users found problems. You can try and let's see if it works for you :).
User avatar
japh
Euro-Master
Euro-Master
Posts: 1471
Joined: Tue Nov 11, 2008 1:26 am
Location: Dalkey, Ireland

Re: EBTST - EBT stats tool

Post by japh »

Having registered and uploaded my csv files I logged out, however, for reasons unknown, I am unable to log in again.
Post Reply

Return to “EuroBillTracker Statistics”