The other day, while automating our load tests, (you heard that right – we are even automating load tests as part of agile efforts) i came across an interesting scenario.
Load runner, which we where using to automate the clustered web application, spread all its output across tons of small files, containing one/many similar lines. The 2-3 lines each file contained would look something like this
Report.c(199): Error: App Error: refresh report. DataSet took longer than the refresh rate to load. This may indicate a slow database (e.g. HDS) [loadTime: 37.708480, refreshRate: 26]. [MsgId: MERR-17999]
Report.c(83): Error: App Error: report took longer than refresh rate: 26; iter: 32; vuser: ?; total time 41.796206 [MsgId: MERR-17999]
But what we wanted to know to know at the end of the test was a summary of errors as in
- Report.c(199): Error: App Error: refresh report. DataSet took longer than the refresh rate to load. This may indicate a slow database (e.g. HDS) – 100 errors
- Report.c(200) Error Null arguments – 50 errors
Approaching the solution
How would you solve this problem ? Think for a moment and not more ….
Well being the C++ junkie that i’m, i immediately saw an oppurtunity to apply my sword and utility belts of hash-tables / efficient string comparisons etc and whipped out Visual Studio 2010 (i havent used 2010 too much and wanted to use it) and then it occured to me
Why not use Java ? I had recently learned it and it might be easier.
The Final Solution
cat *.log | sed ‘s/total time \([0-9]*\)\.\([0-9]*\)/total time/;s/loadTime: \([0-9]*\)\.\([0-9]*\)/loadTime: /;s/iter: \([0-9]*\)/iter: /;s/refresh rate: \([0-9]*\)/refresh rate: /;s/refreshRate: \([0-9]*\)/refreshRate: /’ | sort | uniq –c
I probably should optimize the sed list of commands such that i can pass in the replacement commands as input to sed from a file. But i left it at that, happy that my solution was ready to use / would work all the time / does not require maintenance / saved me a lot of time.
Thinking about the episode, i was reminded of the hit Japanese samurai movie Zatoichi and its happy ending at the end.
- The villain (problem) is dead
- Samurai is happy with his effort
- The subjects (files/compiler/system/software/hardware) are happy due to the efficiency created
- Karma (company + job) is happy due to the time saved.
and to boot it nearly follows the Art of War zen which advices the clever warrior to win without pulling out his sword …
Happy coding !!!