Wednesday, 24 May 2017

Mother Russia....

Добрый вечерDobriy vyecherGood evening




I have so many viewers from Russia at the moment, I thought I'd say a big hello to you all... How are you all navigating here folks?  Do let me know.

I see some served in from Yandex, some from pages on Reddit, however the vast majority of Russian based viewer stats are just direct references, so somewhere my pages are clearly syndicating...

BTW, I'm pushing 11,000 viewers a month now folks!  Combined with passing the 1000 post mark this is a glorious place for me to be in! THANK YOU ALL SO MUCH!

Tuesday, 23 May 2017

RIP Roger Moore

Today has gone from bad to worse... Aside from the tragic attack in Manchester, for which I'm sure we all send out heart felt thoughts, I also want to send my little thoughts to the family of my favourite Bond.... Sir Roger Moore


I think it perhaps fitting tribute that just this weekend we enjoyed his dashing portrayal of our favourite double 0 in "The Man with the Golden Gun".

Sunday, 21 May 2017

1000....

Welcome to my 1000th post... I'd like to thank everyone who appears on these pages.

You are one of nearing one thousand unique visitors each day to my various posts on this blog...

Now, if only you all left a tip... Hmmm...

Friday, 19 May 2017

Football : Too Arrogant to use the Technology!

I enjoyed this from the BBC, however, I have to say Mr Nevin uses sage words almost throughout.

Save for when he says "Too Arrogant to use the technology"... Quite who he is referring to is not clear, UEFA, FIFA, the FA?... Perhaps all of them, whomever it is (or was) we're yet to see this technology, until 2018's World Cup...

Take a look yourself.

Monday, 15 May 2017

Development : The Cathedral & The Bazaar

Recently I had to explain to a none-developer how we had been working on a project, that is to say he and I.  I explained that our common boss worked very much in terms of the Cathedral, everything is big, grand, old, with set sources for supplies and almost dogmatic adherence to their ways of working.  Whilst he and I had been working, with extraordinary success, in a much more agile manner, in the manner of the Bazaar.

Anyone could bring items to us, he could filter some queries, I could filter some others, between us we then worked in an Agile manner to achieve a result.

I myself ran the development, and I pushed builds, lots of builds, releasing at least twice daily.  Any problem was picked up, worked on, solved and the next build had it, leaving a train of progressively better builds step by step, until we sit today with a fully functional system and we await input from multiple other sources.

The main source for things we are awaiting is the Cathedral, we have waited a long time, seven weeks of hard work by ourselves far out stripped the expectations of the Cathedral model of working, and once we superceded that model it was too heavy, too slow, too stiff to catch up with us.

The net result is somewhat of a pocket of confusion, work was demanded, it was delivered, yet now it sits unused.

The bazaar delivered, whilst the Cathedral struggles.



("The Cathedral & the Bazaar" is a famous musing upon software development from the perspective of old slow methodologies to the exploding Agile methodologies which suffuse the Linux development and open source development movements).



Saturday, 13 May 2017

Story Time : "Younger Man" with Sleep Apnoea

Not dissimilar to the description J K Rowling gives to the character of Uncle Vernon in the Harry Potter series, I am a "Big Beefy Man" with "not much neck".

As a result last year it became blindingly obvious that my waist was expanding, I also personally noticed a drop in my day-time alertness.  Initially I would like to point out that I have always been a night owl, and it was not uncommon for me to stay up late and get stuff done (especially programming things) and then sleep later.

This fit well with my employment offering flexible start hours too, and so happily for very many years, I got slowly fatter and fatter, sitting down and eating too much, and I slept later and all was merry.

Except, it wasn't... I suddenly noticed I had become extremely moody, irritable, bad tempered, my concentration was dropping and I found it nearly impossible to remember new things.  Initially I bit back my bad temper, and to compensate I started to use a series of physical notebooks and then scheduling apps to help abate my failing memory.

This crooked self-dillusion carried on for about four weeks, the wife later told me I was a nightmare to live with, however at the time I could not explain the reasons for the change, I could think I didn't want to act that way, I didn't want to forget my lunch, or eat junk, or be a bastard.  But I was being, and there was nothing I could seemingly do about it.

So, I took a few days to think about what I was doing, what had changed, I was very tired, I had to admit.  I kept going though... Kept going, caffeine, pushing black coffee down me, and sugar, lots of sugar.  I never felt I was eating well, but I needed the energy, without it I faltered, stumbled and one day I said "no caffeine today, lets see how I get along".

Driving home one moment I was at red lights, a blaze of other tail lights a head of me, the next blink, I was alone... Sat at the lights still, no-one around me... I had fallen asleep at the wheel.

I literally pulled over and slept there and then.

I contacted my GP, and said "I think I have sleep apnoea", this notified the DVLA, I suspended driving and got one with treatment... Unfortunately this took sometime, so my coping mechanism in the meantime was to sit up in bed more, and to fit a black-out blind to the window.

This alleviated symptoms, but I knew I wasn't sleeping deeply, a short time later and a night of sleep monitoring it was confirmed, I have obstructive sleep apnoea.

I thought this was because I was fat, because I was getting older.  But it seems thats not just it, I am large, and all my family are.  However, the tiredness makes you want to eat more, the BBC has recently covered this phenomenon more than once, and I have to admit it was nice to hear.

I was issued with a CPAP machine, was immediately sleeping so much better.  It was like night and day.  And for the last year I have enjoyed this machine immensely.

However, the straps for the masks go slack over time, as they go slack your sleep falters again, as a man if you have too much facial hair air escapes the mask (have a beard but very short trimmed it's not a problem), far more of a problem is how the mask  seal skirt wear and then the straps go slowly slack.

I would have to argue that the company making these (Phillips) could be held somewhat to account for the latter, as it seems to happen at a predictable, almost designed rate, however I have to concede that the material used has certain requirements and the felt like vencro fastening straps with their slight elasticity are pretty good, but the continual re-stressing of the product does result in quite quick deterioration in performance, even before you take into account the biological factor (sweat & skin cells, and having to then wash the straps).

However a year on, I'm not wondering whether the machine is enough for my condition, I've noticed more than just slack straps, more than wearing seal skirts, more than just a bad nights sleep that more days than not I am quite tired of a day.

This is not everyday, but perhaps two days every fortnight or so, there's no obvious reason for this biologically, environmentally there's no changes either.  I have to admit to slightly more stress in my life at present, however pressure or stress have never stopped me sleeping before (a side effect of studying the martial arts - if a bloke bigger than me can hit me, a bit of code not working in the office is nothing to sweat over).

I will have to discuss this with my consultant in early June.

Being an every inquisitive soul I have started to look at other factors into why I might have apnoea whilst my brother (a close body match to me) does not, and as far as I know no-one else in my family - despite our having a string of heavy snorers - doesn't have apnoea.

One thing I mulled over, before doing some reading, is that during my mid to late teens I had orthodontic treatment, I had four healty pre-molar teeth removed and the two following molars behind these pulled forward.

I've just read, in two sources, that these molars play a role in strengthening the stiffening the sides of the soft pallet, and they also provide a framework onto which the throat muscles structure, then of course the net result is that your jaw is shrunk, leaving less room for the tongue.

I have to be completely honest here, if I let my tongue rest in as natural a position as it can, as easy feeling as it can be, it sticks out over the end of my lower front teeth.  But sits slightly behind my upper bite.

I have the distinct feeling that if I had not had this procedure, yes I would have slightly wonky front teeth, and yes there may not have been space for my wisdom teeth to fill in but my tongue would fit inside my teeth.

As a consequence of this poor fit I have to draw my tongue slightly backwards, with a kink mid-way, to let it sit behind my lower teeth, add the very nature of the action of the flesh being slack in my apnoea and this is a recipe for real trouble.

Having made mention of my weight problem, I did grow whilst without my CPAP machine, I went up to 21 stones around July 2016, by December I was 19 stones, and now today at posting, I am 17 stones and 9 pounds, you can go convert that yourself.

I'm still big, but I feel so much better, even with a few crappy days sleep I feel so much better.

Coming up in June, lets see if my consultant agrees with my orthodontic question, and if I need anything corrected or improved with my CPAP machine.

Wednesday, 10 May 2017

Development : Python, MySQL and Protocol Buffers Build

Today I've come to a totally virgin installation upon a server, this was for a work group I've got to head up whom are looking at pushing MySQL with Python.  And things initially went wrong...

I stipulated they had to use Python3 and thought everything else would be fine for them to install with Pip3, so...

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install python3
sudo apt-get install python3-pip
sudo apt-get install mysql-server

Everything looked fine, their user could install packages through Pip3 and they had Flask and a few other things and were flying.  I used the mysql client on the command like to test I could add a few items and also ran...

sudo mysql_secure_installation

To do the basic hardening of the database, so everything was fine... Right?....RIGHT?!?!

No, not exactly.... "I can't get mysql.connector".... Came the cry.

And they were right, it reported a myriad of build issues and could not install.  I took a look... NIGHTMARE!

It appears the installation of mysql.connector via Pip3 depends upon Protocol Buffers from google for the latest version of mysql.connector... Which the Pip install didn't sort out, at least not easily... Luckily I run a whole gaggle of virtualized machines, so I could quickly spool up a new instance and try a few things out...

This is the script I came up with....

cd ~
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install -y python3-pip git autoconf automake libtool curl make g++ unzip
sudo pip3 install flask
hostname -f > hostname.txt
sudo apt-get install -y mysql-server
sudo mysql_secure_installation
sudo ldconfig
cd ~
git clone http://github.com/google/protobuf.git
cd protobuf/
./autogen.sh
./configure
make
make check
sudo make install
sudo ldconfig
cd python
sudo python ./setup.py install
cd ~
sudo ldconfig
sudo pip3 install mysql-connector --install-option='--with-protobuf-include-dir=/usr/local/include/google/protobuf' --install-option='--with-protobuf-lib-dir=/usr/local/lib' --install-option='--with-protoc=protoc'

Lets go through this step by step...

cd ~
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install -y python3-pip git autoconf automake libtool curl make g++ unzip

This is just the basics, so our installation will now depend upon git, autoconf, automake, libtool, curl, make, g++ and unzip, most all your standard distributions will contain packages for these, we use -y just to skip any permission requests.

sudo pip3 install flask
hostname -f > hostname.txt

The next step is simply a couple of items for our project, we're going to use Flask to provide a restful interface, and the "hostname.txt" is simply to remove our need to call "hostname" again later.

sudo apt-get install -y mysql-server
sudo mysql_secure_installation

Our next step is to install and secure the MySQL service.

sudo ldconfig
cd ~

Generic code now, to simply reload the library list and change to the home directory.

git clone http://github.com/google/protobuf.git
cd protobuf/
./autogen.sh
./configure
make
make check
sudo make install

This is the build of protocol buffers from google, so we pull it from their github repo, we move into that folder, prepare and configure the build, then make the whole build.  By far this is the LOOOONGEST instruction, on a single core 1GB equipped virtual instance this took around 45 minutes.

Once complete we simply need to reload the libraries again...

sudo ldconfig

However, protocol buffers are still not installed within Python, so we are still in the "~/protobuf" folder, we now need to go deeper, into the python folder and perform the setup installation under python...

cd python
sudo python ./setup.py install

When complete we again need to reload the libraries...

sudo ldconfig

And the final, secret sauce, is to actually install mysql connector through pip3 with protocol buffers...

sudo pip3 install mysql-connector --install-option='--with-protobuf-include-dir=/usr/local/include/google/protobuf' --install-option='--with-protobuf-lib-dir=/usr/local/lib' --install-option='--with-protoc=protoc'

This is a single command, spanning one single line.

And voila, once complete you get to use mysql.connector in your python code...

import mysql.connector
import gc

con = mysql.connector.connect (user='whatever', password='something', host='localhost', database='yeahyeah')

con.close()
con = None

gc.collect()

You can find out more about why I nullify and garbage collect a connection in my previous post.

Monday, 8 May 2017

Software Development : Get Constant in C++

What does "const" mean to you?  Does it just mean a value can not be changed?  If so, you may want to read on!

Const is one of those things, which though not unique to C++ is given more meaning when you leverage C++ fully, const allows you to not only define a value as invariant, but also to instruct other programmers coming to your code how a value should be used, how when it is passed as a parameter it should be treated and ultimately how to protect data from needless alteration, de-synchronisation or simple corruption.

Const is therefore your friend, and if you've come to C++ from C, Java, C#, Python or one of the other myriad of languages which don't treat const with as much relevance as C++ you may want to read more than I can say on the topic. 

Bjarne Stroustrop (the inventor of C++) and other authors on the topic (notably Scott Meyers & Herb Sutter) explain in much more detail than I ever could, but for brevity here are two examples of const from my own coding standard which I implore you to digest.

1. Initialise Const Values in the constructor ONLY....

Any instance of the "Ex" class can now access it's "Pi" value, and we communicate to all viewers that the value is a constant and only edited in one place.

2. Where-ever possible pass values as constant references....

We know for certain that the "p_Radius" parameter being passed is NOT to be changed by the function, this is important when you are thinking about letting code document itself, and easily accomplished with "const", and especially annoying when using a language lacking const!

Whatever you do, use const consistently, this needs no specific explanation here, however if you begin following my second rule, you must stick to it!  Don't change half way through a program, if you import a third party API which is not using const in the same way, abstract that third party interface away with a series of your own shims, to make it correct for your usage (the compiler will optimise all these layers away) to leave you with consistent and more easily maintainable code.

You can find must much more about const-correctness elsewhere, I'd start here.

Thursday, 4 May 2017

Software Engineering : My very bad way to set up makefiles

I may have wrote this post, but I'm just being lazy... Read on.

I've made other Boost videos, and other codeblocks videos, and I've always assumed that the user at the other end was well aware of how to compile & build code.

However, it seems there are lots of users whom are not aware of how you actual create a program from source code, at least not with C++.

So I'm going to show you the absolutely most wrong basic way of creating and using a makefile to perform a build...

The first step of C++ program creation, is that trivial matter of creating the source code, easy right?... Yep, so lets skip that step, and go to when its once  and we want to pass the code through a compiler, the compiler turns the source not into a program, but into an intermediary format called an object file or object code.

The compiler then leaves the process and another program takes over, this second program is known as the Linker, and it takes the object code and links it with all the system calls and other libraries you are wishing to use.

There are other steps involved in the process of building a program, however, compiling & linking are the two main steps; when I began programming you were made very much aware of this by the compiler and linker generally being such large complex programs and the libraries being so big you had to physically change floppy disk between the different steps of the process.

This process was also taught to me in school & later college, it was common practice to understand a compiled program was actually compiled then linked.

Unfortunately, this knowledge seems to have fallen in to the way side, it is of course more important to learn how to program than be bogged down in minutia, however, once you have learned I strongly believe you should expand your knowledge and really have a holistic knowledge of the craft of programming.

So, back to our program, the make file, what might they look like? Well they are simply text files,  which contain commands very similar to bash script... We call this make file script by passing it to the "make" program, you can get make for Linux, mac or alongside MinGW for windows.

Lets start writing a make file for any basic C++ program, which links the Boost libraries we've built somewhere on disk, and which looks like this:


#include <iostream>
#include <string>
#include <boost/filesystem.hpp>


int main (int p_argc, char** p_argv)

{
     if ( p_argc > 1 )
     {
          std::cout << "File or Folder ";
          std::string l_pathString(p_argv[1]);
          boost::filesystem::path l_path(l_pathString);

          if ( boost::filesystem::exists(l_path) )
          {
               std::cout << "Exists!";
          }
          else
          {
               std::cout << "Not Found";
          }
          std::cout << std::endl;
     }
}

Save this as "main.cpp" and then define a variable called "FILES" into which we place this filename.

FILES=main.cpp

We want to start our make file with a variable which contains the compiler we wish to use, for Linux on my test machine I'll use "g++".

CC=g++

So now we have a value called "CC" which is a string of characters with the value "g++".  The dollar sign indicates that "CC" is a variable within the script, which we might want to use later, I use these kind of variables throughout makefiles so I can place at the top any paths or references I like, or I can alter them when I move the makefile from Linux to say Windows, without needing to change hundreds of places throughout the script, I can change just one location.

I find this working with strings is 95% of what working with makefiles is about, next in the file we then define the strings for where to find both the boost headers and the binary libraries are...

BOOST_INC=/home/xelous/boost
BOOST_LIB=$(BOOST_INC)/stage/lib

Notice, I create the root folder for boost as one variable, and then in the next line I use that variable to extend into the "stage/lib" folder, so I never duplicate the root folder for the boost libraries.

We can now define at least our build command, slightly different the command is a verb, in this case "build", which is the command we will use here, but which is also the default.  If you save this makefile as "makefile" in a folder, then navigate your console to that same folder and type "make", the make program will look for "makefile" by default, find it and build the "build" verb by default!

build: 
   $(CC) -I$(BOOST_INC) -L$(BOOST_LIB)

So, what is this command doing?  Well it is saying to use the compiler, with the Include folder for boost and look for libraries in the boost library folder we've set.


The "-I" and "-L" parameters are the same parameters one would use on the command line for "g++".  This command therefore equates to "g++ -I/home/xelous/boost -L/home/xelous/boost/stage/lib", but I think you will agree it is somewhat easier to manage.

We also need to tell the program to link against some libraries, and that we want them linked statically in this case.   So our next variable will be called "CFLAGS1" to pass the Compiler flags to the command.  Back at the top of our make file I therefore add....

CFLAGS1=-fpermissive -Wl,-Bstatic

This file tells the compiler to be permissive of some common errors and treat them as warnings.  You can use whatever parameters you wish for your compile, the flags available and their meaning are quite varied, however you will need to use a second set to within the build command, so lets define that now....

CFLAGS2=-static-libstdc++ -std=c++14

The second tells the compiler and linker to link against the static versions of the libraries we select later, you will need to look at the boost invocation pages in order to go back over our previous posts and build boost static and take advantage of this option.

I'm not going to explain static linking here, nor any of the other switches, available.

Now we're linking the C++ standard library, and boost, statically we need another variable with the list of libraries to link, on the command line to g++ this is with the "-l" (lowercase L), except with we are going to make the program threaded, so we can set up a new makefile script variable called "LIBS", into which we set the libraries, like so.

LIBS=-pthread -lboost_signals -lboost_system -lboost_filesystem

We now have the compiler, the folders, the libraries, the flags... What about the target file for the build?... Well I want it to be an executable called "
fredrick".

TARGET=-o fredrick

Now we need to put this all together into a command, just running the script with the variables does nothing, we need to define a verb... Lets call this "build" a verb in the makefile has to have a colon following it...

build:

And we can tell make what to perform for a build:

build: 
   $(CC) -I$(BOOST_INC) -L$(BOOST_LIB) $(CFLAGS1) $(FILES) $(CFLAGS2) $(LIBS) $(TARGET)

We might also want to clean the project, so we'd want to delete the target file maybe, by adding another verb:

clean:
   rm $(TARGET)

We can then perform the call "make build" or "make clean" in the folder with this makefile and perform our build.

A bit rough and ready, but it works to get you up and running.

Saturday, 29 April 2017

Jobs : Value My CV

I just ran across this site which takes your CV (probably data mining everything about me) and it extracts your education, your experience and your personal insight to generate a value for you.

I've just put my CV through this, here's the Skill Cloud....

it's not exact, I know for a fact "Comics" is not on my CV anywhere, B2B software isn't either... BiLingual is a stretch as I speak poor GCSE school French... And a bit of English...

But the value it's placed on my CV is interesting....

That is a lot more than I earn.... A LOT more...

How valid their valuation is, what metrics or heuristics they apply isn't made clear, it might just be a simple look up and tick the box followed by summing a set of weighted values.  But for my location this is not a realistic estimate, no-one would pay this for me here.

Based out of London, New York or anywhere in California then maybe, but here in the middle of England.  No way.

But I take this that my current employers are getting an absolute steal employing me... Not least as I just noticed I'm 37 hours unpaid overtime... Hmmm, gotta sort that out.

Thursday, 27 April 2017

Server Admin : Ubuntu 17.04 thinks it's Ubuntu 12.04???

Yeah, I'm serious, I've taken time tonight to look at the release of Ubuntu Server 17.04, specifically to set up a new mini-server which is to be Core 2 Duo powered and on 24/7 as boot strapper & service strapping server itself.

But, before I run I like to walk, so I set up a 2 core, 1GB RAM VMware machine from the 17.04 ISO... Take a look at the first thing it has presented to me....


Yes, just read that again... I booted the server... and the only action I took was to log in... Welcome to 17.04... All good...

What wait?.. Why am I being warned to upgrade my 12.04?  This is 17.04?

Before I ran around like my last vestiges of hair were on fire, I decided to do a simple test, I've previously found that Ubuntu often goes wondering off on the internet for message of the day (motd) information, so I pulled the network card (virtual) out of the machine.

This results in a long boot time, but you at least know no remote files or services are going to be listing things on your screen...


Five minutes later, I get to see what the system says... From experience I think Canonical have shown some news and up to 80 characters of 40 lines of news, I've never seen that much, but it has been a while since I looked at their motd scripts.

After logging in, I still got the message, however, a fresh install (without any network) didn't show the message, so I believe the install we see here cached something from an online source.

Anyway, taking a look in the /etc/update-motd.d folder, you can see a series of numbered scripts, these are so numbered to allow Canonical, or yourself, to add message of the day scripts, and keep them in the order you see them.


Checking "00-header" we see just the usual log in.

Then "10-help-text" is the three lines about documents, management and support.  I actually add "#" to the start of each of those lines to remove that files actions, I don't delete the file though, just in case.

The next line "50-motd-news", this looks to be the culprit... I'm not even going to look inside the file, because I can see the next file in the folder is "90-updates-available" and I can see in the login that the updates available happen after the message I want to be rid of....

So this strange, confusing, message is in "50-motd-news", I'm going to cut to the chase and kill that file.

And now my login is much neater, I have added a call to "ifconfig" into the 10-help-text, but my login is now clean of this strange message.  But I'm not impressed this has gone on, and I'm going to have to take a look through all these other motd scripts to see what and where my server is going off to.... Hmmm.

Monday, 24 April 2017

Tech Office: Talking Dress Code

Lets put this clearly, I don't believe in a set dress code (as in defining what anyone can wear) in technology, I've seen anecdotal evidence of the history of IBM where they insist on suits and sock garters and alsorts of things, I've seen companies demand you wear a suit - indeed I used to work in Formalwear production and had to wear a suit - I like suits, however, I don't believe they have a place in the everyday development office.

If you're seeing customers, if you're "forward facing" then sure, dress up.  But smart casual is enough for me, what is "Smart Casual"?

Jeans, sure.  Shirt, sure.  T-Shirt if it's plain, sure.  What's not... Well, how about a 30 year old sagging woolen jumper?  Please no, just no.

What about a bright orange flair neck shirt which has its top three buttons missing and is worn by a hairy chested 1970's throw back?  Please god no.

What about shoes?  Well, I wear a nice pair of Rockport leather shoes, they're functional, comfortable, ageing but decent looking... Trainers, I have no objection to trainers, sandles... No... No sandles.

Shorts?... Please No, just no.

Combat trousers, I'm find with them, so long as you're not putting ammo in the pockets, oh and they're a plain colour.

Then you come to the more sticky points, its kind of the dress code, and sure medical considerations aside, how do you smell?  I've worked with people, not directly in my team, who stink.  Bodily, as though they don't wash, but also as though they've been dipped in something... Chemicals, paints, cigarettes, bad breath, BO... It's just not acceptable, it's not polite, when you're working elbow to elbow or if you smell so bad people give you a wide birth, or close their mouth as they come past your desk, beware you will be asked to spray.

That said, what do you spray?  Please god, nothing totally stinking...

Thursday, 20 April 2017

Sys-Admin/Dev Ops : Assumption is Danger

As a systems admin, or dev ops, or whatever your job title might be, never ever assume that the person you're handing a system to has a clue.  This might seem harsh, but it's true, and proves itself true time and time again.

"Assumption is the mother of all f**k ups"

About a year ago I deployed a system which automatically sent requests to remote machines (via SMS) getting those machines to report their status or send back error information, but also to gather some basic information.

It has run happily for a whole year, it has been all pretty plain sailing, the hours and hours of work I put into it, to automate it and keep it self-sustained have paid off, zero faults, zero down time, self-regulation is the way forward for me; even if it took slightly longer to put the system in place, it has needed no human input for nearing a year!

However, the unit needed to move, about a week ago, it needed be physically picked up and taken out of my small server room and into the official server room, a dark cupboard basically controlled not by myself or my cohort, but the IT boffins.

Fine, I notified the customers, went off to the IT area, sorted out who I was to hand it to and physically delivered it to the chap, I watched him start to plug it all back together, power, wires, boot, fine....

I assumed he'd do this seamlessly....

Until this morning, well a morning last week, as I post these with a date in the future.  That morning was hell, I walked into a wall of customers not being able to get to their machines, the Easter weekend was looming, performance needed to be monitored, customer sites didn't have regular staff, explaining to temporary cover staff that system would be off was not a prospect I relished. 

To be frank, a lot of flapping going on, more than I expected... IT reported the system back online, but customers didn't stop flapping... Indeed, none of the estate seemed to be able to connect in... 1 hour, 2 hours, I've asked the boffins to check it time and again "It's fine", they tell me.

I look locally, I can't see the controller machine on the network, I can't see it through the remote management console... Where the hell is the machine?

I assure the customers I'll have answers within the hour, I hit social media with the same, this is going very public, and I'm rather annoyed as for a whole year things have run seamlessly; but been ignored, now its offline for a scheduled purpose and everyone is complaining, I do not want my success wiping away in a flood of negative press.

I call the IT boffins... "we'll look into it"... No, no no, you'll get onto it right now, not look, not glance, answers are needed.  Action from you is needed before my Re-Action goes nuclear.

I wait, five minutes, I was willing to give them ten.... My phone rings...

Them > "Hello?"...
Me > "Answers?"...
Them > "Yeah, you know when you brought it back?"...
Me > "The Machine?"....
Them > "Yes"...
Me > "I remember, why?"...
Them > "Well, it has power"...
Me > "Good"....
Them > "Not really"...
Me > "Why not?"...
Them > "Because that's all it has, it's not been plugged into the network"

I hung up.  They plugged it into the network, I had a slew of data come through... The customers were pacified.

I however was not.

I've had an on the spot review, firstly the IT bod who did this was held to account, second I was held to account for not noticing.

In not noticing I admit that having had it run cleanly for a year I had turned off the performance reports and I admitted I had assumed a network machine being handed to an IT bod would be plugged into the network.  People were not happy, least of all me, but that was the fall out.

However, I then had to do a tertiary clean up and after the Easter break I spoke to three of my main customers, trusted operators, the actual folk who should have been using the machines at the remote sites; not temporary staff; I asked them why they had not noticed.  The replies...  "Because it had worked for so long without an issue", "like you make it work, so we just guess it always is" and "we didn't notice it was offline".

They were very much putting everything into my court, assumption on the part of all parties was to blame.

The lessons learned for me are to now keep checking, keep monitoring, use my automation to report status, to fix faults and if human errors creep in, to let me know.

I'm now off to spec up a service I can run on one of my own servers, just to ping the network machine which went AWOL and receive a report from it to let me know what its up to, this might be a bit of python or just bash on a cron task, but it's going to be something rather than nothing.

I will NOT assume again.

Tuesday, 18 April 2017

Software Development : Failed to get Agile

I've just been party to a conversation about a project elsewhere in my work place, my team is not involved, I was observing passively (alright, alright, I was ear-wigging).

The conversation was quite heated, one member of staff was adamant things were fine, whilst another was adamant they were inadequate.  The two of them were at complete logger heads. The driver of the conversation ran like this:

"We're not really designing software, we're asking everyone's opinion, writing it all down and only picking the things we really need to do"

As an agile developer this is essentially how I run my team, we write every possible item down, everything and I weight them, schedule them and during out sprint hand-overs we reorg whom is going to tackle diffing parts of the system to share the experience and share different things.

This chap however, was incredulous... He expressed "WRITING EVERYTHING DOWN" as a bad thing... He only wanted to do the things he felt fit, he wanted to sit down and look at the specification, produce an analysis and ONLY do what he suggested.

This would have made perfect sense to be towards the end of my academic study of software development; before the reality struck home in the work place, and I was flabbergasted to hear this chap simply working twenty something years in the past.

I mean, he's old... This company is old... But, not that old surely?

I literally caught myself tipping my head to one side as if trying to pour those words, and the way they were said, back out of my brain.

He didn't stop there though, he sat and without knowing it essentially dismissed as absurd the complete concept of Agile development; at least Agile as I use it...

"You'd be constantly juggling which task to do next, swapping people on and off tasks.... What would you do?  Meet daily, what would be the point?"

I'm not sure whether this was genuine inflexibility or purposefully derailing the effort to adopt agile beyond the scope of my own team, whichever it was, it sounded and felt extremely awkward.

It makes me wonder quite if anyone outside my team actually uses Agile processes around here...

Sunday, 9 April 2017

Server Admin : How Good is your Backup?

How robust is your back up solution?  Go on, be honest with yourself, how good is it?... Because I've seen a whole host of them and, at this very moment, this is the screen up on one of my servers....


Yes, my raid 5, just a test raid 5 with three really bad recycled SAS drives in it has failed; this doesn't surprise me, but it does delay me because I now have to rebuild the data... However, I know my data is good.... Lets see how good my back up is.

This back up is coming from a DD created raw image of the virtual disk, stored to and soon lifted from my NFS accessible ZFS mirrored back up server.

Therefore you would be right to ask, why are you rebuilding the virtual RAID disk in the above screen shot?  Well, I'm going to test my back up strategy!

I popped the known bad disk and the good disks out, replaced all three and I'm able to test a restore to a new virtual disk set, I have a USB boot drive ready, this is a test.

This kind of test, a real live restore, is sorely missing from so many enterprise set ups, so ask yourself is your back up going to work?

Wednesday, 5 April 2017

Development : Anti-Hungarian Notation

Whilst cutting code I employ a coding style, which I enforce, whereby I output the scope of the variable being used with a prefix.

"l_" for Local
"m_" for Member
"c_" for constant
"e_" for enum

And so forth, for static, parameter and a couple of others.  I also allow compounds of these, so a static constant would be:

"sc_"

This is useful in many languages, and imperative in those which are not type strict, such as Python.

Some confuse this with "Hungarian Notation", it's not.  Hungarian notation is the practice of prefixing a type notification to the variable name, for example "an integer called count" might be "iCount".

I have several problems with anyone using Hungarian Notation, and argue against it thus. With modern code completion and IDE lookup tools this is really not needed, with useful and meaningful naming of your variables the type is not needed and finally there are multiple types with the same possible meaning... i.e. "bool", "BYTE" and "std::bitset" are they all 'b'?  What about signing notation, so you compound "unsigned long" as "ul" to the name?

It all gets rather messy, a good name is enough.

However, the scope of the variable might change, the scope might not be enforced, and in none strict languages you might have a variable go out of scope and then automatically re-create the value with a blank value, if you don't follow your scopes.

Therefore I can justify my usage and enforcement of this coding standard.

What I can't stand however is when someone listens to my explaining this, they read my coding standards document, they even go as far as having me reject their code during peer review for these reasons, and then they dismiss my comment with the "it's just Hungarian Notation"... Scope is not type, and type does not define scope, don't be fooled!