Programming


The not so new VS 2010 has added what seems like a cool feature in the solution explorer window. It shows all your external dependencies

But wait a minute,  what is all this that you see when you expand the external dependencies ?

The drop down shows ALL the dependencies, even those that are indirectly pulled in. The feature is useful but i wish someone had put a thought into how it is used … If i had been shown the layered dependencies, that showed the directly referred h files and then the ones that are further pulled in and so on, it might have been more usable.

Wish someone from Microsoft does hear this and make amends !!

Advertisements
Some time back i encountered a scenario, in which a chain of stored procedures gets called recursively and this was eating into our performance (90% cpu time). All of a sudden obscure micro issue about whether to use table variable or temp tables suddenly became important.
After ferreting out some good sources of information on comparing these, we managed to reduce the cpu consumption by
  • Making all our stored proc use temp tables rather than table variables made a difference for us, because our procedures where getting invoked frequently and stored-procedures are recompiled every time if table variables are used.
  • But we had minimal impact from db writes due to temp tables because very few data was actually used
Ultimately the maximum improvement came from changes to the algorithm to make sure that the procedures do not get needlessly invoked or at least parts of them do not run (recursive algorithm was flattened out)

LearningIf micro optimizations start to look important, it is time to look for improvements in the algorithms that are used.
NOTES – Information gleamed from the exercise
  1. Most often you will require table variables
  2. Table variables cause FEWER recompilations than temporary tables. (source MSDN)
  3. Table variables seems to be scoped at the stored proc level. So if you create a table variable inside a while loop, it would retain old data (painfully found from a bug)
  4. Table variables cannot have indexes created on them except the primary key specified when creating these tables
  5. Create index command can cause query recompilations
  6. Table variables are held in memory and involve no disk IO – therefore no rollbacks are possible on these. On the upper side no locks are acquired by using table variables.
  7. Table variables cannot have statistics created on them so for huge data it COULD be inefficient  ?

An internal mail on the cost and success of our agile implementation efforts, describe the cost of a single bug as x ……person days …

Finally, change !! 🙂

I was reading the VC++ team blog ten minutes back and took their poll, inputs from which would go to improve the next Visual Studio release.

The interesting question (one of many) from the survey, which prompted me into writing this post is as follows –

How much of your time do you spend on each of the following?

  • Defining the problem
  • Requirements gathering
  • Designing solution
  • Writing code
  • Building code
  • Refactoring code
  • Debugging code
  • Writing tests
  • Testing
  • Deployment Support

The options against each are, (Not at all, some, About 50% and A lot). There can be overlaps. So what would be your answer?

Here is mine –

  1. About 50% time designing the solution
  2. About 50% time writing code (Base code writing time)
  3. About 50% time refactoring it  (I always keep improving the structure)
  4. some time testing it    (I write a lot of functional tests to test my code)
  5. some time debugging (I pride myself in lowest bug counts ever in most teams i ever worked)

What does your distribution look like?

ps : If you are in big corporation, your time spend overall for coding related activities above might be only 30-40% of the overall time you have.  This poll is meant to measure the activity spread within that little amount of time you really get to work with the code.

Last week i had some spare time which used in creating a tool to parse JUNIT reports and produce automated reports. This was done to do away with the stupid recurring responsibility which the team members used to get every once in a while, to baby-sit some JAVA project builds .

However since this was a pet project and done on my personal time, i could not get any QA time on this or afford to spend the normal time i would spend in creating the code normally. Not with the new Agile process and the accompanying rally tool used to measure all my tasks.  So  i  did a real quick job out it and ended making some parts sloppy and corrected a few not yet encountered bugs, which i had to rework, since after all i could not afford to have such a sloppy tool lying around with my name on it.

It was then that i thought i should perhaps list down the origins of the bug which i saw in the code, along with the other common ones i have seen over the years in different code bases. So here they are, without much ado.

  1. An unfinished running list of “things to do” like cleanups / code smells, which introduces an unmanageable level of code debt.
  2. Heavy refactoring, which moves code around, which changes post and preconditions for the code copied, which is not handled or ignored in its new area – this was the new one for me.
  3. Lack of adequate error checks
  4. Not performing required cleanup when you exit a function or block
  5. Performing multiple things at a time WHILE coding (2 code-bases / support calls/ emails / calls / meetings …etc)

Bugs produced from this category of reasons are the most common and perhaps the most easiest to sense early, from a casual read of the code. For the same reason therefore, it is totally irresponsible of a team or its management if code which smells of the above manages to get released.

The next category of bug origins, would lie on a higher level and are more difficult to address and cannot be found from glancing the code and would typically require good code checking tools like lint or good reviews.

  1. Incomplete information about the API being used (eg free has to be combined with malloc and not new)
  2. Logical mistakes in the program flow

Seriously though, any programmer worth their salt, would have learned enough of their tools and would have passed enough theoretical exams, to make the above two points, extremely rare in the real world. It is here that the importance of the ‘frivolous’ reasons, increases in significance. In all my programming career, the most bugs i have seen are due to initial set of 5 reasons, which makes it all the more important to eliminate those.

It is also  true, that only if the first set is eliminated, will the second become more obvious in code reviews, which is another reason why the initial set MUST be caught and corrected.   The majority of the cost of a programming project,  which is incurred after the coding is done, (80/20) over its support cycle, is also perhaps a good reason why these frivolous yet frequent bugs must be rooted out of the code base.

The remaining 3 reasons which complete our top 10 would be

  • Platform complexity (custom Operating Systems, rare databases / applications, messy legacy code etc)
  • Architectural mess
  • Missing or wrong or Incomplete Requirement

Of-course the last few are issues  that cannot be easily corrected once inherited and one can but put up with the mess and keep chugging along, managers and programmers included. In such conditions it is obvious that one need not add to the woes by creating more bugs and unwelcome noise on top of an already complex platform.

Therefore by all means and for gods sake create CLEAN code the first time !!!.

Spot the bug

Code A

STDMETHODIMP CMIB2Iface::CollectData(LONG DeviceId, BSTR IpAddress) {
	list<CString>::iterator listIter;
	for(listIter = ccmList.begin(); listIter != ccmList.end(); listIter++) {
		CString ccmIP = *listIter;
		long tempCCMID = getDeviceID(ccmIP);

		// Handle it only if device ID is valid
		if (tempCCMID != -1) {
				CString component;
				long eventType = 0;
				if (endPointType == 0) {
					eventType = 47;
					component = "Gateway Link - ";
				} else if (endPointType == 1) {
					eventType = 48;
					component = "Unity Link - ";
				}

				component += endPointName;
				// Construct an event.
				LunaEvent ev;
				CString attributes = "linkDestination = " + endPointIP;
				ev.attributes = attributes.AllocSysString();
				ev.eventTypeId = eventType;
				ev.deviceId = theCCMID;
				ev.component = component.AllocSysString();
				ev.timestamp = (COleDateTime::GetCurrentTime()).operator DATE();

				try {
					if (endPointStatus == 0) {
						if (tempCCMID == theCCMID) {
							// Raise/Update an event whenever status is 0
							if (prevEndPointStatus == 0) {
								// Case 00
								unsigned long result;
								MY_LOG_DBG(functionName << ": Updating Event on Component : " << component);
								if (faultManager->updateEvent(ev, &result) == S_OK) {
									MY_LOG_DBG(functionName << ": Updating Event Secceeded");
								}
							} else {
								// Case 10
								unsigned long result;
								MY_LOG_DBG(functionName << ": Raising Event on Component : " << component);
								if (faultManager->raiseEvent(ev, &result) == S_OK) {
									MY_LOG_DBG(functionName << ": Raising Event Secceeded");
								}
							}
						}
					} else {
						// Case 11 and 01
						// Always clear the event, and for all CCMs in the cluster.
						unsigned long result;
						MY_LOG_DBG(functionName << ": Clearing Event on Component : " << component);
						if (faultManager->clearEvent(ev, &result) == S_OK) {
							MY_LOG_DBG(functionName << ": Clearing Event Succeeded");
						}
					}
				} catch(...) {

				}

				SysFreeString(ev.component);
				SysFreeString(ev.attributes);
		} // End of if (deviceID != 0)
	} // End of for loop
	return 0;
}

Code B


STDMETHODIMP CMIB2Iface::CollectData(LONG DeviceId, BSTR IpAddress)
	{
	this -> m_sLog.SetIpAddress(IpAddress);

	/*   ---------------------------------------------------------------------   */

	m_oInspector.AddCollatedRowInspectorFn_Post(FilterIfTypeInfoCollect,
						(void *) this);

	/*   ---------------------------------------------------------------------   */

	m_oInventory.ClearAllCollectors();

	if (BOMBED(DoTableGenericInit(m_oInventory, sIfaceTblColln, m_sLog)))
		{
		_log(L_ERROR, "Mib2 collect table init failed \n");
		return S_OK;
		}

	/*   ---------------------------------------------------------------------   */

	if (PASSED(m_oInventory.CollectInventory (DeviceId, IpAddress)))
		{
		GetIfFaultsExistForDevice(DeviceId, LINK_DOWN_NUM , m_iFaultExist, m_sLog);
		m_oInventory. InspectData(DeviceId, eForInventory);
		m_oInventory. SaveData   (DeviceId, eForInventory);
		}

	return S_OK;
	}

Code Review Department

Code Review Department

If tasked at reviewing the 2 pieces of code listed above, chances are that, you might as a reviewer, want to get rid of the task of reviewing A, than spend any more time on an obviously bad effort at coding. Having badly styled code around, allows bugs to remain, and fixes to be nothing more than half hearted patches which neither the patcher nor the original coder can recognize after the act is done.

One bad turn soon begets another and the rate at which the bad code degenerates into sphagetti mess can only be matched by the speed at which engineers long for a total rewrite of this mess. (Failing which they secretly start yearning for a change of job or worse, stick to the same one in a lost/zombie sort of way or decide what they really want to be is a manager)

Humans are not compilers

Human Compiler

Human Compiler

Unlike compilers, humans being are quite easily put off by bad code and no act of redemption can save a bad piece of code from degenerating further and further until it can only be discarded. This is frequently the reason why engineers advocate re-architecture, which is hugely more expensive than adapting existing code. The lost business oppurtunity that this represents can be huge and the very effort of doing so could run into millions of dollars per year for a major software company like CISCO.

Any act of review on such a bad piece of code can only seen as a sham – who after all in their right mind would want to spend so much time reviewing someone else’s code when it would so obviously take such a long time to do it.

So the first step in getting working reveiws is to not let the process to be viewed as sham – and the only way that this can be achieved is by respecting the reviewer’s time and patience.

How do you respect a reviewers time ?

By giving readable code (see above)

How do you make code readable ?

  1. By avoiding coding smells
  2. bi folwing sm cdng convntns(by following the same coding conventions aka coding style)

Twenty different people and their cousins would have an equal number of vastly differing coding styles. If all of these are to be heaped on same page or on the same reviewer, again its equivalent to implicitly admitting that you have no control and that the whole process is a sham. Therefore getting the same style of code to be used EVERYWHERE is what has to be ENFORCED for successful code reviews to happen.

My experience

In the only place i have personally seen code reviews to actually work, a body of code could be rejected simply because a character exceeded the max allowed no of column width of 80 or because in a single location in code, variable was not aligned. The rationale behind this was the assumption that that the coder did not know about this rule and therefore there would be many other places where this mistake would be made.

The act of rejecting the whole body of code due to single misfitting character sends the right signals and soon it becomes a matter of pride to get code accepted without major comments.

The other advantages in having readable code is that

  1. Simple Bugs are easier to spot
  2. Design issues are easier to spot
  3. Review happens faster (that builds more support for review)
  4. Bug fixes are easier
  5. Module upgrades or re-architecture would be easier
  6. SPotting reuse chances are easier – and hence aids faster development
  7. tois which so often happens are much more easier

How can the organizations aid in creating readable code

  1. Use tools that can perform static analysis based on accepted coding styles and flags issues, before reviewer even reads one line of code
  2. Make code readability indexes, part of statistics of the project so that they can be easily tracked
  3. Use tools that can reformat existing code base to the accepted style, no matter what that is, so that its easy to get off the block when the process starts

1 out of 2 is still only 50%

Fixing the code review-ability is only half the fight. Where would a reviewer be, without a programmer who wants the code to be reviewed ? In a team of colleagues, its only more than easy for everyone to scratch everyone else’s back and implicitly give easy reviews. Its also natural for colleagues to contest one others reviews. Therefore, there has to be an external entity and stimulus to engage in code reviews.

Of-course enforcing the code standards using tools part of the build process and enforced through management numbers would be a good start, though that alone might not make it stick.

My experience is that usually stimulus takes the form of bragging factor or nerd pride amongst peers. When an obviously respected person  / geek is to review your code, you would want it to be more shiny and that extra perfect.

Companies like Cisco can easily make this happen by having more tech leads to review the code, and having a system of recommendations where recommended parts of code can even get promoted to technical evangelists like Stroustrup or Gosling once a year for review, which can be such an honor. Tracking  programmers, coding bullshit index would also be a good measure for wanting a review.

The human factor

Coding and reviewing code, is ultimately a very human activity and any mechanical solutions to the problems in this domain are bound to fail. Ultimately a more humane and social solution would be required to transform this bit of ‘work’ into something more than statistic, and something of an advantage.

“Any fool can write code that a computer can understand. Good programmers write code that humans can understand.”

– Martin Fowler

Having been a full-time C++ programmer for all of the past ten years, I recently had the unique opportunity of seriously studying Java almost 10-15 years after it started becoming popular. In the meantime I’m also playing with Python on a personal note. This affords me a unique perspective, of having seen the past and at the same time looking at two new languages with a fresh eye.

C++ vs Java in 2009 = Assembly vs C++ during the 1980’s

The impressions or the vibes that i get in this new learning process is eerily similar to the same doubts and choices i felt / read about during the initial stages of my education, while playing around with assembly, C++ and Visual Basic.

Needs assembly

Needs assembly or C

Assembly was THE choice if speed and memory choices was paramount and nothing else mattered but it was damn hard to use.  All the old timers claimed you had to know assembly to understand whats happening on the system and that C++ was only for productivity improvements. They lamented about how new minds never learned / worked seriously with assembly and hence where at a disadvantage when it came to figuring out what really happened inside the box.

However at the time (1980) C++ was fast catching up, even embedded systems where moving to C++(maybe without using the templates feature).  Java currently is in the same state that C++ used to be in 1980 wrt assembly and there are many who clamour for C or C++ to be taught in universities rather than Java for the sake of better understanding.

How can  Java be as fast as C++?

It is not. However the hardware that is around now is vastly superior to 1980 and 1990 hardware, which made Java seem slow THEN, in the same way C++ seemed slow in all those small devices compared to raw assembly or binary code. Our current crop of laptops are more powerful than the servers used during the time when Java was beginning to get popular. This improvement in hardware makes the difference in speed highly immaterial for all practical purposes, because human beings cannot make out the difference all that much between 500  and 50 microseconds.

Fast forward to 2010

Is as fast as a 1999 computer and runs Java

Is as fast as as 1999 computer and runs Java

C++ is what you require if you need pure speed like in the critical loops or core libraries. You could substitute the word Assembly instead of C++, in the previous line, had this post been written 15 years back.

But Java is more useful from a business point of view for application development due to increased productivity and more features.  It is therefore no coincidence then that embedded devices like mobile phones (and even dog collars, according to Java books) are now running Java. In fact the last Nokia mobile phone that i bought was guided by the factor that it had a data port using which i could upload custom Java program onto it.

Java vs Python = C++ vs Visual Basic

When C++ used to be THE top language around, and assembly was gradually being kicked to the ground, there was something else in periphery, which was Visual Basic and all other RAD languages, which basically gave you forms based development and automatic memory management.

Internal IT or back office apps need not be cool

Internal IT or back office apps need not be cool

However they were nowhere as fast as C++ and hence where relegated to internal IT applications, which never needed to be as slick as professional, cool looking  applications and could be developed fast enough.

Fast development is very critical when it comes to small shops with less resources or IT departments on a shoe string budget. Many business applications certainly fit the bill and where developed on Visual Basic. It is exactly the same situation now with Python when comparing it to Java.

Python in 2009 = Visual Basic in 1990

Developing using Python does definitely seem like a breeze with not much hassles around object orientedness or strict rules around exception declaration or big infrastructures. Is Python as comprehensive as Java, with as many no of apis as Java has ? Probably not, although it is fast catching up in popularity, like Java before it, caught with C++ and  that is saying a lot. There is no denying the fact that Python and Ruby would soon have many programmers who are trained in it by choice.

Will run Python as fast as Java

Will run Python as fast as Java

I have not compared the performance of Java vs Python. But i assumed Python could be slower from what i have heard on the web. Please do correct me if iam wrong on the above aspect.

If the economy keeps growing like it did in the last ten years, we would soon have quad core or even eight core desktops a common place and would not have to worry about Python’s performance compared to Java. I just hope folks don’t introduce messy co-orporate stuff and make it verbose and bulky as Java before it.

3  cheers for easy coding !!!

Next Page »