Friday, 17 March 2017

Development : Managers Trust your Developers

I have been so busy, I'm deep in the middle of a new system development with work, but as a personal side line I'm also completely re-engineering a different system, you know when you've worked on something or in a field as long as I have it becomes second nature to want to do things your own way... So I've been doing just that.

I picked a coding standard & style, I've stuck to it, I've pulled in SDL and Boost, it's all C++ and it's around 100x faster than the original system and amazingly more simple to access.

The problem, the boss doesn't want it, he doesn't trust it.  More importantly he doesn't trust me.

My previous boss has seen the system he's gobsmacked, his opinion was that it was exactly what was needed four or five years ago, and to have done it alone with no support in parallel with my real daily work just blew him away.

So what was my current bosses argument?  Well, it's one I can't counter, his point is that the current system I want to replace has been out on the market with real customers for years, nearly twelve years in fact, and I ported it to cover five different platforms.  He's right, it is, it has been, and it will be again, but that doesn't make the system any good.  All it lets a manager in his position do is quantify the risk versus reward, and I can not argue or knock him for that.

However, what I can knock is when you have an employee, like myself, who lives and breaths this technology, who writes books and blogs on the topic, and they've worked for you for 14 years... They're investment in the product is vastly more than the product risk for taking up the new system, especially when the new system would only go to trusted customers for trials, it would not immediately go out to the full estate of customers.  You reduce that risk you've presented drastically, there is less risk, but the reward for introducing that new system is vast, acceptance of it is equally as likely as giving customers the original old system in a new skin.

They are 100% compatible by the way, content for one works on content for the other, they work in the same sequence and order, they are the same except the underlying code is vastly better in one and a squelchy pile of over cooked spaghetti in the other...

So, to managers out there, the world over, yes evaluate your risks, but your developers are a huge component of that risk, if you have a good one, trust them when they do 40+ days of leg work on something, they're not doing it just for their own sake, they're doing it because of a driving need.

Wednesday, 8 March 2017

Development : Visual Studio Giving False Reports of Missing Symbols

I have just run across an issue which has me slightly baffled... Sending the build down to my automated server and it started coming back as failed, panic ensued, so that I started to then do builds on my local Visual Studio instance... (which is VS2013). 

Now, this code is perfectly working, has been fine, released and tested for... Since about 2011.

This is how the error shows up...


Taking a closer look...


We can see there are suddenly a bunch of unresolved externals, specifically the definitions of a bunch of functions.

There is NOTHING wrong with this code, indeed, the work around to get a build to complete is to comment out any one of these functions....


Rebuild, and it now fails on that specifically missing function body.


Finally, re-enable the code and do a further rebuild... and everything completes perfectly...


Consulting the internet about this one, as everyone here has drawn a blank, and one person suggests that this is a miss-match between the Byte set setting of the project (Multi-byte) and any libraries being Unicode.  However, I have no third party libraries... The mystery only deepens.

A full fresh check out of the code, still fails, a full clean fails, a rebuild fails, rebooting the machine has failed.  None of the fixes has made a difference, and more problematic is that the failure happens sporadically.

Monday, 27 February 2017

Gaming : Ark Survival Evolved


When the neighbours are bigger than your hut...


You can't dig a defensive moat....


You can only watch them hump the rocks...


And your camp mate falls unconscious with his legs tucked up his bum!

Ark Survival anyone... Dedicated PVE Server coming soon....

Wednesday, 22 February 2017

Development : Scrum & Leveraging Virtualisation (My Hobby Team)

Recently in one of the Scrum teams I oversee as their Scrum Master, I saw a common problem, one or two members of the team were being blocked by mundane issues once or twice a week.  For one member of the team this was a physical PC fault, for the other it was disk-space.

They are running their own equipment, outside the remit of my authority or the company control, therefore I had to clear the decks in some manner, the solution?

Well, the development was all Linux, I created a standard Ubuntu image and issued it to all concerned and asked that these be the ONLY virtual machines run to complete work.

The first week of this resulted in a slow time, I found people were not using; as per the original specification; virtual machines at all, I found they had a mish-mash of development materials, and importantly nearly every person asked me to allow them rights to install software.

I didn't allow this, instead I used the Scrum management software I've put together and collected all their ideas about editors, tools and software to add to the image, and we had an additional 10 minute open discussion going around the group to assess the merits of each piece of software, what it would be used for and how it might leverage either an easier time to task completion route or how it might introduce potential benefits in other ways.

I started with the proposer of the software, then alternated between those in the group whom had indicated they liked the software option and those that had not, in cased where no-one objected to a piece of software, I simply took the concensus benefits and added them as points below each item listed.

In the end we had reduced the pile of suggestions to three, the first a lightweight text editor, many were split between "just using nano, pico or vi" and those "desparate to use sublime".  This came down to a simple choice, one was free, one was not, we have a sub-zero budget since having to source those drive caddys last month... Everyone was going to use my preference "nano".

The second was a performance measuring tool option, to monitor the system, I opted to leave this decision to the testers in the group they have yet to advocate a choice, but I decided to take the option out of the scrum teams scope of interest, their code has to be fast and that's the end of that discussion.

The third was an art package, there were many options, one person wanted to do all their art in Paint.NET on a windows machine then move it into the main project, another wanted to create assets with GIMP and another wanted to use Photoshop. on his Mac, an easy decision GIMP.  It's native to Linux and free.

The standards were set, we would use tar and gzip on the image I had issued, Nano, GIMP, Code::Blocks and the compiler was the GCC for the group.

For diskspace, problems, every individual member was issued a brand new 1TB drive, the Ubuntu Image and they had to host this virtual machine however they wanted, it needed a single CPU core and 1GB of RAM minimum.  Many of the developers were able to tweak and give the machine 4 cores and 4gb of RAM.

Since then the team has run a lot more smoothly, issues with your software, pull the VM image down again... Issues with the source tree?  Delete your local working copy and resync with the repo server!  Need a place holder image?  Fire up GIMP.

This was the first time, since the New Years, that I had reviewed the team in any depth, as a Scrum master in this situation I don't have a lot of blocking or shielding duty, individuals work in their own time and in their own homes, so they have to deliver or fall short.  It is a hobby project after all.

However, our first Sprint feedback is in from yesterday evening, the overwhelming feel is that the team is now working much more quickly, they are verging on out stripping all previous schedule expectations (which were secret from them) and yet they all feel this is easy gliding development time, moving from product point to product point without the hassle of their machine or not being sure how to run an item, or not having the same set up.

My next task?  To do the same with the tester group... What a bag of cats they might be....

Saturday, 18 February 2017

There's a Rat in me Shed, What am I gunna do?

We've had a rat come visit our shed...

Well, when I say paid a visit, I mean he has dug a complex trench system for the war which I am now perusing against him...


This was beneath a patio, and somewhat is a surprise... The first signs of a visitor were these droppings in the shed where the guinea pigs live...


I spotted these and immediately knew they were rat... The curled one on the right is a classic shape for rat droppings...

Anyway, I then decided to take a look around, and I found a black smudge and some trailed mud, in the back of the shed where no-one could go, so it was obvious there was a visitor, I moved everything out of the shed and found a neat hole cut into the back wall... Bastards!

I baited this and nailed it up, then placed more bait inside, after I had jeyes fluid cleaned the whole area and swept up.  As you can see in the second picture there was a lot of dropped matter, bedding and bits from the guinea pigs, but there was also hay from the stables in there and horse feed... Bad place to store them in the bag, rather than dumped into the hard plastic bins we have.

Anyway, inside secure, I set about looking outside, and saw what I thought was a run behind the shed in line with the hole, so I baited that and blocked the main exit into the garden with a tall slab.

I also saw this darker area of soil... Now we have a rabbit who lives free in the garden most of the day, and I figured she had just been moving earth around, as rabbits tend to do... Maybe for a fortnight I've seen this earth moved each evening, and assumed it was the rabbit.

But, it wasn't, seems the clever ratties decided it was a bore to go all the way around the shed, and they cut the above tunnels beneath a storage box... I lifted this and found the tunnels....

I baited the one nearest the shed, blocked another with yet another slab on end, filled in the tunnels then laid new slabs on them and placed the storage box back on this new footing.

The detritus in the middle of that run is all stuff from the shed, mainly guinea pig feed... So we've clearly been spilling it... In fact I know I have, the perfect invite for these little bastards.

Anyway, poison down, earth moved, slabs laid and gnawed shed patched up... The guinea pigs however are spending the night stood under three layers of blankets in the garden as I put so much cleaning fluid down the shed hasn't dried yet.


Thursday, 16 February 2017

People : Co-Workers We Remember

Today everyone has some form of social media, at least in this business, LinkedIn, Facebook, twitter or this blog!  They're all examples of being able to keep in contact with people out there, either bi-directionally in the case of LinkedIn & Facebook, or uni-directional in the case of Twitter and this blog.

I like the latter, I like people to be able to find me; if they know me, or just discover me.  I recently had a message from a chap involved in my first Rack Mount Mistakes posts.  His name is Stuart, and he was a very nice chap, he helped with the clean up and was my second when we had to confront and appoint the blame for the debacle on the chap responsible.

He contacted me to say "hey, I remember this"... Which is exactly what this blog is about, but I had to tell him I didn't clearly remember him...

After a little discussion we came to a conclusion, he found me to be a pain in the bum, fun and good at my job, but I was young, much younger than him, but in charge; which he didn't like.  I was early twenties, he was early thirties.  Now he's entered his fifties he looks back on those days with fondness and even said "I thought back then I had the nous to do your job, that you were pushy, but I know now I could never had been like that with people, never told them what to do and get those jobs done when I was that age".

Two things struck me about this, firstly that he was complimentary, but also I realised why I didn't remember him clearly.  He was one of those rank and file kind of guys he did his job, he fitted in, and he got the task done.

Because he never rose above the surface level he was never a nail head needing me to drop on him like a hammer...

To my fault this made me place him in my "of least concern" perimeter, so as time has gone on I've forgotten that he was diligent, forgotten that he was a good developer, forgotten even his surname.  This is good and bad.

However, taking this tail of thinking further, I realised I remembered lots of the bad ones... The bad egg, the nail which poked up and needed hammering into place, the ones who didn't get on with their jobs, the ones who disappeared from their desks for hours on end, or who missed deadlines, even some whom even today I chase around bad code they've left in their wake.

I don't contact either group of ex-co-workers, very few of them can directly contact me, perhaps this is good for the bad ones, but it's certainly not ideal for all those good ones I've met and worked with over the years....

Andy S
Max B
Paul D
Leon B
Dave P
Ravi S
Steve W
Tony
Andy K
Richard (Dad!)

And the valiant few I still work with and like to work with where I sit today, I salute you all.... Anyone not on the list... I either forgot about you or I don't like you; you decide which!


Tuesday, 14 February 2017

Programming : Python MySQL Connector Debug

Today, I was asked to look at a server for a friend, their problem... "It just stops working after a few days"... A few days turned into "between three and five".  Doing some mathematics I found they had between 125 and 350 unique visits to the server, each unique visit represents one customer or one remote unit of their fleet.

They relay their data from these to individual database instances on one MySQL Server, so there is about 30 customers each with many unique databases.

The problem?... Well, I find this very distressing, as they open one connection for each arriving remote client, use it and then they closed it... Right... RIGHT?!??!!

import mysql.connector

l_total = 0
while (True):
    # Count
    l_total += 1
    l_res = l_total % 100
    if l_res == 0:
        print (l_total)

    # Open a connection
    con = mysql.connector.connect(user='root', password='***', host='localhost', database='Pickles')
    cursor = con.cursor()

    # Query
    query = ("SELECT * FROM VeggiePatch")
    cursor.execute(query)

    # Retrieve the data
    cursor.fetchall()

    # Close the query cursor
    cursor.close()

    # Close the Connection
    con.close()

This is my test code based on the way their production code works, as having read the error log I see the problem is in the connector constructor and delves down into the networking code.

This of course crashes after around 33,000 cycles.

They're not willing to change their script "willy-nilly", I in fact think they're petrified I've found this problem.  Googling around I don't find any official explanation of this error, only anecdotal forum posts about the MySQL Connector not cleaning up after itself and so reusing the sockets fails over time.



The better solution is to garbage collect the connection each cycle...

import mysql.connector
import gc

l_total = 0
while (True):
    # Count
    l_total += 1
    l_res = l_total % 100
    if l_res == 0:
        print (l_total)

    # Open a connection
    con = mysql.connector.connect(user='root', password='***', host='localhost', database='Pickles')
    cursor = con.cursor()

    # Query
    query = ("SELECT * FROM Tickets")
    cursor.execute(query)

    # Retrieve the data
    cursor.fetchall()

    # Close the query cursor
    cursor.close()

    # Close the Connection
    con.close()
    con = None

    gc.collect()



I also tried to garbage collect each time I printed the the "total", each 100 passes, but this still crashed, the fixed loop here has so far done just under half a million cycles without issue....