Monday, 26 June 2017

Computing Education 2017

Here in the UK there have been several waves of trying to educate new generations as to the art of compute science, this started when I was a boy with the BBC Computer Literacy project and concluded soon after with a drought of interest from non-technical educators and politicians a like through until fairly recently.

The BBC reports that there has been a low amount of uptake of new Computer Science GCSE studies.

And I can believe this, however the neither the BBC nor government seems to even point as to why this is, they talk about pupil disengagement or lack of interest.

I however contend that the government and educators and indeed the BBC completely fail to spot the elephant in the room, kids study not for jobs or skills, however they do study what is emphasised, IT has always been an "also ran" topic, it's not Maths, nor English nor seemingly as important in appearance as any other topic out there.

In my day this was the case because few understood computing, today however it seems IT is still an unimportant subject, today it's seen as unimportant because of it's ubiquitity.  Kids see easy to use computers, they walk around with them on their wrists, on their pockets, they are the always online generation.  If they need now know how to do more than turn the wifi on why should they care?

Likewise the educators see using a modern computer as easy, so why should it be a subject of study really?

And finally, the basest of problems, is the employers, if employers are willing to employ a programmer who studied horticulture, why should one study computing?  If the employer will take on an IT support operative who has no qualifications but whom is handy with a screw driver and knows how to plug the right parts of a PC together, then why should they bother to get formal qualifications?

Ultimately, for computing to be taken seriously, you need a passion for it, you also however need a reason to study it, and until that reason exists in the form of accessible all tier jobs that actually require a formal computing qualification there is little hope in pushing back up the chain to educators or government that computing is important and needs to be studied.

Friday, 23 June 2017

Development : My Top Three Testing Tips

I've said before, and I'll say again, I'm not a fan of Test Driven Development, tests and test frameworks have their place, but they should not; in my opinion; be the driving force behind a projects development stream - even if it does give managers above the dev team a warm fuzzy sense of security, or if it allows blame to be appropriated later, you're a team, work as a team and use tests on a per-developer basis as a tool not as a business rule.

*cough* I do go off topic at the start of posts don't I... *heerrhum*, right... Top Three Automated Testing Tips... From my years of experience...


1. Do not test items which are tested by masses of other developers... I'm talking about when you're using a frame work of library, ensure you are using it correctly certainly, do this at training or with your coding standard, but then do not labour the point by re-testing... Lets take a good example of this, the C++ Standard Library.

The Standard Library contains many collection classes, these classes have iterators within, lets look at a vector:

#include <vector>

std::vector<int> g_SomeNumbers { 1, 3, 5, 7, 9 };

We could iterate over the collection and output it thus:

int g_Sum(0);
for (int i (0); i < g_SomeNumbers.size(); ++i)
{
    g_Sum += g_SomeNumbers[i];
}

However, this is not leveraging the STL properly, you are introducing the need to test the start poing "int i(0);" the end condition "i < g_SomeNumbers.size();" and the iterator "++i", three tests, slowing your system down and complicating your code base.

int g_Sum(0);
-- TEST SUM START
-- TEST i COUNT START
-- TEST RANGE CONDITION LIMIT
for (int i (0); i < g_SomeNumbers.size(); ++i)
{
    -- TEST ITERATION
    g_Sum += g_SomeNumbers[i];
    -- TEST SUM CALCULATION - THE ACTUAL WORK DONE
}
-- REPORT TESTS

Using the iterator, we leverage all the testing of the STL, we remove the need to range test the count variable, we remove the need to test the condition and leave only the step as a test to carry out...

int g_Sum(0);
for (auto i(g_SomeNumbers.cbegin()); i != g_SomeNumbers.cend(); ++i)
{
    g_Sum += (*i);
}

Our code looks a little more alien to oldé timé programmers however, it's far more robust and requires less tests simply because we can trust the STL implementation, if we could not thousands, hundreds of thousand of developers with billions of other lines of code would have noticed the issue, our trivial tests show nothing of gain, so long as we've written the code to a standard which uses the interface correctly...

int g_Sum(0);
-- TEST SUM START
for (auto i(g_SomeNumbers.cbegin()); i != g_SomeNumbers.cend(); ++i)
{
   -- TEST ITERATION
    g_Sum += (*i);
   -- TEST SUM CALCULATION - THE ACTUAL WORK DONE
}
-- REPORT TESTS


2. Do now allow values which have been tested to change unexpectedly... I'm of course talking about "const", which I have covered before on these pages, but constness in programming is key.  The C family of languages allow constness at the variable level, you may notice in the previous point I used a const iterator (with cbegin and cend) as I do not want the loop to change the values within the vector... Constness removes, utterly, the need to perform any tests upon the integrity of your data.

If it's constant, if the access to it is constant, you do not need to test for mutations of the values.

Your coding standard, automated scripts upon source control submissions, and peer review are your key allies in maintaining this discipline, however it's roots stretch back into the system design and anlysis stages of the project, to before code was cut, when you were discussing and layout out the development pathway, you should identify your data consider it constant, lock it down, and only write code allowing access to mutable references of it as and when necessary.

Removing the need to retest mutable calls, removing the need to log when a mutable value is called because you trust the code is key.

In languages, such as python, which do not directly offer constness, you have to build it in, one convention is to declare members of classes with underscores to intimate they are members, I still prefer my "m_" for members and "c_" for constants, therefore my post-repository submit hooks run scripts which check for assigning to, or manipulation of "c_" variables.  Very useful, but identified by the coding standard, enforced by peep review, and therefore removed from the burden of the test phase.


3. Remove foreign code from your base... I'm referring to code in another language, any scripting, any SQL for instance, anything which is not the pure language you are working within should be removed from the inline code.

This may mean a stored procedure to store the physical SQL, rather than inline queries throughout your code, it maybe the shifting of javascript functions to a separate file and their being imported within the header of an HTML page.

But it also includes the words we ourselves use, be that error messages, internationalisation, everything except code comments which is in whatever language you use (English, French etc etc) should be abstracted away and out of your code.

Your ways of working, coding standards, analysis and design have to take this into account, constness plays it's part as well, as does mutability, where ever you move this language to, and whatever form it takes test it a head of time, and then redact that test from your system level tests, trust you did it right based on the abstraction you've performed.  Then avoid burdening your system throughout the remaining development cycle.

One could expand this to say "any in-house libraries you utilise, trust their testing" just as I stated with the STL in my first point, however, I am not talking about code, I am talking about things which are not code, which are uniquely humanly interpretable.

The advantage of removing them and pre-testing the access to them is that you retain one location at which you have an interlink, one place at which a value appears, one place where they all reside and so you leverage easily converting your programs language, you leverage easily correcting a spelling mistake, and all without needing to change your system code; perhaps without needing to even re-release or re-build the software itself (depending on how you link to the lingual elements)

Ultimately reducing the amount of testing required.

Wednesday, 21 June 2017

Development : Coding Style Clash

I've spoken on these pages before, and even shown, that I generally code to a standard, one of the rules I have it NOT to use Hungarian notation, but to use a notation telling me the scope of a variable, and then give it a meaningful name.

Now, over the last few weeks I've been involved in a new piece of coding with a group of like-minded individuals.  And what getter way to explain this oxymoron than through our coding standards.

Now, I like to use "m_" for member, lots of people can accept this, I like "p_" for parameters, a lesser few reject this than you'd think and some people even quite like this as they suddenly get to reuse their useful meaningful names, and in languages like Python they suddenly get to distinguish between locals, globals and parameters really easily... But then the controvertial one, the one which causes me most angst.

Locals being represented with "l_"... You might say the lower case "L" is asking for trouble, and I would agree, but I'm sticking with "l_" locals, and so did lots of other folks, however... I said we're a group of like-minded individuals... So not a group at all.

And so locals caused a bit of a flare up, because another chap had simultaneously, and without any prompting of mine, also come to the conclusion to use this same naming convension, however with one difference... He didn't call locals locals, he called locals instances, and so he used "i_".

Yes, that was it, the whole difference... "l_" to "i_"....

I could live with this, and when coming to look at his code I could read it perfectly well, and I was more than able to leave it alone, he however, could not, and indeed would not, leave mine alone.  He went so far as to write a script which line by line, looked for "l_" and replaced it with "i_" whenever he pulled from the remote branches.

He then proceeded to commit this massive change, the result... His one liner alteration was lost in a swamp of about 35,000 changes in his check-in.  Making his contributions near impossible to track.  I asked him to keep this just to his branches, and to stick to the "l_" for the master or pushes he tagged for merging, but he simply would not listen, he changed every "l_" to "i_".

Trivial things like:

std::mutex m_TheLock;

void Somefunction()
{
    std::lock_guard<std::mutex> l_LocalLock(m_TheLock);
}

Became:

std::mutex m_TheLock;

void Somefunction()
{
    std::lock_guard<std::mutex> i_LocalLock(m_TheLock);
}

And the change logs got longer and longer, the tracking of bugs became harder and harder, until I decided to pull my support of the project, and this is important, not because it looked like a hissy-fit on my part.

But it demonstrated to me something fundamental about the project, I happily bent to others, I let them name and title things, I didn't fix their code, only stuck to my own, I published my coding standard and people started to adopt it within the group, but I never forced it upon them.  Yet this one chap wanted to import and force his will through this tiny insignificant thing.

So, always, no matter your personal feeling, stay flexible, don't be that guy who insists on something for zero gain (or in this case much loss).

[Foot note: If your project/group/team has a manager, a leader, and is not a collective as the above, please disregard this, have a coding standard, make it clear, efficient and update it regularly, but have ONE person enforce it, don't let two tribes go to war over "i_"].

Monday, 19 June 2017

Bash : Power of Pipes

Subtitle: "Get IP Address Easily"

When I say easily, I mean not so easily, but with the proper tools... Let me explain, it's been one of those days... I've a remote server running some flavour of Linux, and no-one knows it's remote IP Address, they all SSH into the box run "ifconfig" and note down the value, they then plug this into a config (or worse still were baking it directly into some code) and running their services....

The trouble of course being, years later, they're no-longer the programmers nor maintainers of this machine, I am...

And to be frank whenever the IP address changes I don't want to recompile their java code, nor use vi to edit the various configuration files, I want a script to at least update the settings automatically.

I therefore changed their code to load the IP address, not hard code it, and used some other scripts to put the IP address into the config file at boot...

The first line of that script is what I'm going to document here... so it starts:

#! /bin/bash
ifconfig | grep inet | tr ' ' '\n' | sed -u '/^$/d' | head -2 | tail -1 > ipaddress.txt

This script gives me a single line of text with the IP Address in it, for the one and only adapter in the machine, if you have multiple adapters you'd have to play about with the grep inet to select the row you want with a head & tail call before moving into the  final location, or whatever...

I wrote this up however, and immediately started to use the IP address.

The net result was a request to explain all this functionality to a colleague... Here's what I came up with.

ifconfig gets us the adapter information...
grep strips off lines we don't want only giving lines for the inet adapter
translate turns spaces into new lines
the sed call removes the blank lines, giving just the IP address and some guff
the first adapter IP address is therefore always the second line of this output
we select the first two lines with head
then select only the latter of these two with tail
and write this to a file

Her reply... "What are the Lines?"....

"What lines?"...

"These | things"....

"They're pipes, I'm piping the information from one program to the next..."

"Oh"

"Do you know what pipes do in Unix and Linux?"

"No"

I sent her to this video... https://youtu.be/XvDZLjaCJuw?t=5m15s

Thursday, 15 June 2017

Development : Peer Review Pillock

I don't generally stand by the idea of peer-reviewing all code, not at least until a product is passed from it's original creation pass, that boot strapping has to be all done and over and a certain amount of confidence in the product has to stand before I believe every change needs to be peer-reviewed (certainly group code review internally as you produce that first sprint to a working product - and remember always try to push sprints to working builds, never break things - but don't ask everyone to peer-review every change when there are literally hundreds happening an hour as a system comes online).  Anyway, back to my point...

I assigned my work for review, we're using git, so the pushes upstream get logged and anyone in the team can pick to peer review, the ticket system I'm putting together myself shows whether a change was peer-reviewed or not, but does not make them mandatory.

New developers joining my team however have to spend an amount of time loading the code into their wet-ware via eye-ball MK1, so it's worth them subscribing to the push feed and doing peer-reviews.

And I pride myself on picking the better developers for my teams, it saves me too much time getting them up to speed.... Soooo.... what's my point?   Well..

I just used a Lambda in some C++. The chap peer reviewing it rejected the code.  As he thought that it was gibberish….

auto l_Thread = std::shared_ptr<std::thread>(
    new std::thread([]()
    {
        std::this_thread::sleep_for(
            std::chrono::milliseconds(500));

        InitButtonsAgain();
    });

His comments told me more about his skills than I think he wanted them to...

"Auto is an unknown type at compile time (is it even a type?)"

"Excessive use of std:: use using namespace std; at the time

"[]() is invalid syntax"

"The function braced on like 3 to the last line of the change has no name"

My unfortunate, and impatient, reply was "Update your skills to C++14".

Tuesday, 13 June 2017

My Moroccan Work Week

Many moons ago, when my then boss knew I was the dogs danglies, I used to get sent to work at offices all over the world, and today I want to tell you the story of one of those journeys to and from the "office".

I live and work out of Nottingham, and this one time I had to go work out of Casablanca for a week.  Now, for those of you not aware of this, I do not mean I went to work in a black and white film... Casablanca is a real place, a city in fact, in the North African country of Morocco, exotic... Maybe, if you like that thing.

Anyway, I was in my early twenties and sent on this trip, I spoke broken GCSE French, and was handed a few thousand French Francs (yes it's that long ago, France still had a proper currency, with a history and everything).

The journey began at an indecently early hour, a driver to take myself and a pair of cow-orkers to Heathrow, no big deal, though the driver had a tin of sweets which he was really really over proud of; he was also sceptical I wasn't the son of the two other passengers, as I was so young.

Arriving Heathrow no big deal... The flight fine and dandy... I saw sail boats in the Straits of Gibraltar...

When we landed and were in arrivals we had our bags back, and there was a driver for me, or for my then company... Not the co-workers whom were a different company - merger in progress as it were - so we all went to get in this one Merc... Now, I immediately went to the wrong side of the car, going to the British passenger side, which was the Moroccan drivers side; this baffled the driver.

But getting into the car we immediately found there were no seatbelts... the clip was engaged, but there was no strap.  Asking the driver he said, "no-one uses seat belts here, so we cut them and put the pegs in to stop the car beeping as we drive".

He then proceeded to drive like a loon, on pitch black desert roads from King Mohammed V airport into the heart of Casablanca.  By the time we arrived at the hotel we were ready for a bit of food, maybe a drink, the bar looked at my french francs like I was mad... "Charge it to my room"... silence.

This was meant to be a dry country remember, but the bar was there serving alcohol... Anyway, bed early, a morning start... My suit had just about survived being packed into a back-pack, but someone had taken the batteries out of the side pocket, which had my alarm clock... Sorting out a wake-up call was a bit of a nightmare, but I got there in the end.

Fun fact, order English Breakfast tea, they bring it to you with hot milk... URGH.  I sent it back first time around... I am such a noob.

The fun began mid-week, when I had an afternoon early ending, and got to spend sometime at the hotel, the Hyatt near the Port.  I went for a walk about, and was soon approached by a pair of Moroccan lads who invited me to "come to the bazaar, great café, show you hasheesh"... I declined their kind offer, seeing it as a mix of either illegal, or simply an attempt at kidnap.  he surprised me however by pulling his wallet from his pocket and showing me he'd spent time in Manchester...

The hard week of work ended, and I found myself back at the airport...

But there was no plane... Royal Air Moroc had no plane... There was myself, the two co-workers back from their site, and then one German chap, waiting for a whole 737... Royal Air Moroc however hadn't planned for this.

An hour late they "borrowed" a plane from Air Egypt, and the four of us boarded, this empty plane.

I have no idea why, but the staff insisted that the three of us sit together on one row... a little cramped, whilst the German chap was sat up front waving back at us all alone for the duration.  They really didn't want us to move seats.

The pilot (or it may have been the co-pilot) came back after take off and introduced himself to us, ah the days before they locked the flight deck.  I remember he had a white cotton scarf around his neck, my memory tells me this was on a wire so it stood up like biggles... But I may just be mentally elaborating...


The meal came around about this time... The offer was "Chicken or Goat", seriously... I had the goat, it was nice.

They then wanted to sell us drinks, and I panicked... Trying to ask for a "1664".. In French... instead of just asking for "un bier"... This brain fart haunts me to this day, but I was young, I was an idiot, I wanted to speak French, to a very pretty Moroccan Air hostess... Or maybe she was Egyptian....

I remember that the air hostesses didn't cover their air, a mark perhaps of days gone by?  I don't know, maybe someone can tell me.

The trip was rounded off when I got back to the chauffeur car, to come back from London to Nottingham, and he looked at me and said... "You're not on my list to come back with me".

My co-workers didn't say "lets take him", or anything as I had done in Morocco with the car sent under my company name, oh no, they just got in the car, with me standing there looking (I'm sure) a little green he added "but I'll take you as well".

We set off back to Nottingham.

Other images of Casablanca I remember are of going through the bazaar and a chat in a white tiles store asking me "You want my daughter, she cook, she clean, very clean girl".  She was 12, he didn't mean sexually either, he wanted her to be my house keeper and my take her to the UK.  I politely declined.

I was also accosted from the street, whilst I walked in the Hotel Perimeter garden, a voice from a man carrying a battered looking AK47 and a dead cockerel shouted to me... "Are you American?"  With a distinct twang on the last word.

"No I'm English" I replied...

"English?"  He beamed with his one remaining tooth "I support Manchester United".  I wondered if he was the granddad of the lad who'd accosted me mid-week, I really did.

Good times, quite sad that I sit in an office all the time now.

Thursday, 8 June 2017

Development : No Great Shakes at SQL

I was just handed a technical test, this was for a job interview, and I was a little... well I'll say surprised, but then shocked, by the reply, lets take a look...

I was handed these SQL statements, in several questions, and then told to summarise what I should see on any output of the various instructions....

create table customers (id INTEGER PRIMARY KEY, name Text, age INTEGER, weight REAL);
insert into customers values(73, "Brian", 42, 33);
insert into customers values(1, "Helen", 12, 12.5);
select * from customers;
SELECT name, CASE WHEN age > 18 THEN "adult" ELSE "minor" END "type" FROM customers;
create table orders (id INTEGER PRIMARY KEY, customer_id INTEGER, desc TEXT);
insert into orders values (0, 73, "Apples");
insert into orders values (1, 73, "Oranges");
insert into orders values (2, 1, "Bananas");
select * from orders;
SELECT customers.name, orders.desc FROM customers JOIN orders ON customers.id = orders.customer_id;

I drew up the two tables, their info, the results of the selects and the listing of each order by the last point.

The final question was then, "alter the final query so that only orders by customers named "Brian" are displayed.

My answer was this:

SELECT customers.name, orders.desc FROM customers JOIN orders ON customers.id = orders.customer_id where customers.id = (select id from customers where name = "Brian");

Now, I'm no great shakes at SQL, it's a tool to be used, not a mantra to adhere to, so I understood that I could also solve this problem thus:

SELECT customers.name, orders.desc FROM customers JOIN orders ON customers.id = orders.customer_id where customers.name = "Brian";

Both are valid results, and yes I admit mine results in another query (within the parenthesis) however, in my mind I wanted all parameters directly in a query to be the result of other queries, meaning I could place "Select customers where name is 'Brian'" into a function elsewhere for ease of maintenance, rather than have "Brian" hard coded into the query here.

I discussed this when myself and the technical reviewer went over my suggested solution, I explained "The query in the bracket should be stand along, either as a separate query or stored procedure".

He made a note on his clip board, and I thought nothing more about it.

Fast forward a fortnight, and this morning I heard back from this job, they said I was a very good candidate, but failed the technical test on one key point.  I looked at my notes, nothing looked that key.

I left it there, and went back to the recruiter but asked them for specific technical feedback.

The feedback I received, just an hour ago...

"Over thinks the solution to a problem" and they cited the above answer.

I am flabbergasted, it seems people really would rather my not give a valid answer, or perhaps don't want to discuss the decisions I passed through in order to reach my conclusion.

Either way, I think my solution is valid, my point about being able to move the latter query into its own location is valid, code maintenance is important to me, so being able to break down none-nonsensical ordering things like SQL statements is high on my radar, but to be told I am essentially "too verbose"... I'll be frank, it's pissed me off.

But hey, what do I know?... I'm no great shakes at SQL.

Monday, 5 June 2017

Black border.... KitKat

What have Nestle done?... I see on the packet it says "Now with more Milk & Cocoa", but the chocolate is bitter, catches the back of your throat, and it's not as nice or mellow as it used to be.... What have they done?

It's the end of an era...

The end of KitKat, I doubt I'll ever eat one again...


Friday, 2 June 2017

Tech Job Interviews... Derp

I'm not saying who for, and I'm not saying what for, but over the last three months I've been on a little spree within the jobs market for my kind of career, I've been through phone interviews, technical tests and face to face meetings.

And I've noted a pattern in how things to, they go one of three basic ways...

The first is that the interviewer likes how I hold myself, how I present myself, but they've read nothing about me.  Now this is a large blog, I've written a book, and I publish quite a lot of information about how I work and what I do... To not read this is tantamount to negligence, either on the part of the interviewer or the recruitment folks putting you through for a role.

When I find myself in this situation however it lends itself to my favour, as I can answer questions posed to me, and beware that as thorough as the opposite party aimed to present themselves they've already demonstrated to me that I have to tick the boxes in their filter live, so to speak.

The next is that they've read the information about me but then they don't like me, either how I come across in person, or just some trivial problem like a technical tests; more about those in a moment.  Finding myself in this situation can be tough, for instance having a near four hour interview experience, face to face, meeting not one, not two but three lots of people and being invited to tour the work space, you'd be forgiven for thinking "I think I'm in here"... Just to be told "no" later (yes this actually happened, because I told a chap I don't like TDD, and he frothed at the mouth a little).

The final is where the interviewer has no interest in you as a human being, you have to tick the right boxes, and that's it.  I've had this twice recently, out of very many tests and interviews, one verbally and one via a written form.  Many years ago I graduated, and I walked as a new (one of the first in the UK actually) graduates with a pure software engineering degree... This essentially means I use a computer as a tool, to communicate, to process or gather information, whether I leverage that with C++, Python, Word or just as a paper weight makes no odds to me, so long as the task is done effectively, in a maintainable, secure and under-budget manner.  However, I walked into a room with five other people and was handed an A-Level maths paper... I sat there looking around the dingy room, looked at the paper's title and just got up and left.

This final situation is perhaps the hardest to explain, unless you've been there, you are you, you are a value you are an idea, you are a font of experience and the culmination of your life to date, you are not completing an A-Level maths paper, you are not writing a bit of data processing, you are certainly not some online multi-choice quiz.

But doing these menial tasks proves to someone that you know what you're doing... Apparently?  In the latter case it's almost as though the online quizzed expect you not to just google an answer... I myself would much rather just state "don't know"... but above that I'd rather NOT perform such odd tasks.

If I were to hire staff today, I'd sit down and talk to them, the CV is the main filter, do they have any skills I desire?  Do they have any insight or experience which is of use?  Talk to them.

Then, once I'm happy at the personal level I'd not ask them to perform some monkey tricks, I'd just hand them production code, or a snippet there of... "Comment on this", openly, freely.... The spelling, the layout, the coding approach, any bugs, any suggestions.  But I would not do this without knowing who they are, and never without myself being there, being a personal touch.

I don't know, maybe it's the vogue of today, maybe it's a never ending spiral, but it appears to me that slowly we are divorcing ourselves from one another, online.  This maybe something important to note, the most successful online communities, or sites, involve interacting with one another, take facebook or snapchat as examples, you don't get in there at the ground floor by asking impersonal questions in a static box.  You get the young, the scared, the ones willing to please answering those questions.  The rebels, the technologists like me, we tear up the rule book, we're here to innovate, not tell you where a non-mutable variable is attemptedly being assigned to in a subroutine to make you feel better.

Thursday, 1 June 2017

You know...

You know that moment, when it's been over a week since you blogged, and yet you feel like it's been five minutes as you're so busy... Yeah, I've got that feeling right now...

So I thought I'd take a moment to list a few planned projects I have for these pages, if I ever get time for them....

The first is to look at the Intel TBB library, this is intended to assist in taking full advantage of the multicore capabilities of Intel chips, not something that has ever fully come into my sphere of interested (beyond running threaded code with posix threads or boost::thread or std::thread of course).

The second is some more Javascript, Python and MySQL interconnection code, in order to write a ticket/development control system, this is because I need a new one, I'm a little tired of the ones I use professionally, and the good ones out there on the interwebs are generally paid for, or limited for free, services.  So I want something super simple to handle my tickets... "Super Simple Tickets"... SST... That has a ring to it.

The third is to finally, go through some of my old Atari hardware, I plan to go through a video series just building from all the spares I have an Atari STE and I'm going to sadly sell it, as I have a LOT of junk left around the place.

A fourth might be to put together some AM1 powered AMD based machine, in order to see if the Intel TBB leverages itself smoothly onto that platform, and it appears to be pretty cheap to get entry there.

A fifth is to start recording some more game-play on my brand new EVGA nVidia 1080 GTX SC, which is an awesome piece of kit and has opened up the scope of my game play from my retired EVGA nVidia 700 GTX SC... Yes, Superclocked to Superclocked I have moved, and it was a tear because the 770 still stood up and produced such good results on the games I play, however, Unreal Engine 4 based games coming out over the last year have started to test it, and I never could get good performance in early versions of Ark (before they optimised it) and not at all in the currently unoptimized Player Unknowns Battlegrounds... So an upgrade I had, and nearly £600 later I'm a very happy puppy.  Though the delivery was rather shoddy.

And the final sixth is related to the fifth, to record more about the Medieval Minecraft Fortress and underground themed base I've been building from the excellent book I reviewed previously.

Anyway, that's what I'm planning to be up to... 

What are you guys doing?

Wednesday, 24 May 2017

Mother Russia....

Добрый вечерDobriy vyecherGood evening




I have so many viewers from Russia at the moment, I thought I'd say a big hello to you all... How are you all navigating here folks?  Do let me know.

I see some served in from Yandex, some from pages on Reddit, however the vast majority of Russian based viewer stats are just direct references, so somewhere my pages are clearly syndicating...

BTW, I'm pushing 11,000 viewers a month now folks!  Combined with passing the 1000 post mark this is a glorious place for me to be in! THANK YOU ALL SO MUCH!

Tuesday, 23 May 2017

RIP Roger Moore

Today has gone from bad to worse... Aside from the tragic attack in Manchester, for which I'm sure we all send out heart felt thoughts, I also want to send my little thoughts to the family of my favourite Bond.... Sir Roger Moore


I think it perhaps fitting tribute that just this weekend we enjoyed his dashing portrayal of our favourite double 0 in "The Man with the Golden Gun".

Sunday, 21 May 2017

1000....

Welcome to my 1000th post... I'd like to thank everyone who appears on these pages.

You are one of nearing one thousand unique visitors each day to my various posts on this blog...

Now, if only you all left a tip... Hmmm...

Friday, 19 May 2017

Football : Too Arrogant to use the Technology!

I enjoyed this from the BBC, however, I have to say Mr Nevin uses sage words almost throughout.

Save for when he says "Too Arrogant to use the technology"... Quite who he is referring to is not clear, UEFA, FIFA, the FA?... Perhaps all of them, whomever it is (or was) we're yet to see this technology, until 2018's World Cup...

Take a look yourself.

Monday, 15 May 2017

Development : The Cathedral & The Bazaar

Recently I had to explain to a none-developer how we had been working on a project, that is to say he and I.  I explained that our common boss worked very much in terms of the Cathedral, everything is big, grand, old, with set sources for supplies and almost dogmatic adherence to their ways of working.  Whilst he and I had been working, with extraordinary success, in a much more agile manner, in the manner of the Bazaar.

Anyone could bring items to us, he could filter some queries, I could filter some others, between us we then worked in an Agile manner to achieve a result.

I myself ran the development, and I pushed builds, lots of builds, releasing at least twice daily.  Any problem was picked up, worked on, solved and the next build had it, leaving a train of progressively better builds step by step, until we sit today with a fully functional system and we await input from multiple other sources.

The main source for things we are awaiting is the Cathedral, we have waited a long time, seven weeks of hard work by ourselves far out stripped the expectations of the Cathedral model of working, and once we superceded that model it was too heavy, too slow, too stiff to catch up with us.

The net result is somewhat of a pocket of confusion, work was demanded, it was delivered, yet now it sits unused.

The bazaar delivered, whilst the Cathedral struggles.



("The Cathedral & the Bazaar" is a famous musing upon software development from the perspective of old slow methodologies to the exploding Agile methodologies which suffuse the Linux development and open source development movements).



Saturday, 13 May 2017

Story Time : "Younger Man" with Sleep Apnoea

Not dissimilar to the description J K Rowling gives to the character of Uncle Vernon in the Harry Potter series, I am a "Big Beefy Man" with "not much neck".

As a result last year it became blindingly obvious that my waist was expanding, I also personally noticed a drop in my day-time alertness.  Initially I would like to point out that I have always been a night owl, and it was not uncommon for me to stay up late and get stuff done (especially programming things) and then sleep later.

This fit well with my employment offering flexible start hours too, and so happily for very many years, I got slowly fatter and fatter, sitting down and eating too much, and I slept later and all was merry.

Except, it wasn't... I suddenly noticed I had become extremely moody, irritable, bad tempered, my concentration was dropping and I found it nearly impossible to remember new things.  Initially I bit back my bad temper, and to compensate I started to use a series of physical notebooks and then scheduling apps to help abate my failing memory.

This crooked self-dillusion carried on for about four weeks, the wife later told me I was a nightmare to live with, however at the time I could not explain the reasons for the change, I could think I didn't want to act that way, I didn't want to forget my lunch, or eat junk, or be a bastard.  But I was being, and there was nothing I could seemingly do about it.

So, I took a few days to think about what I was doing, what had changed, I was very tired, I had to admit.  I kept going though... Kept going, caffeine, pushing black coffee down me, and sugar, lots of sugar.  I never felt I was eating well, but I needed the energy, without it I faltered, stumbled and one day I said "no caffeine today, lets see how I get along".

Driving home one moment I was at red lights, a blaze of other tail lights a head of me, the next blink, I was alone... Sat at the lights still, no-one around me... I had fallen asleep at the wheel.

I literally pulled over and slept there and then.

I contacted my GP, and said "I think I have sleep apnoea", this notified the DVLA, I suspended driving and got one with treatment... Unfortunately this took sometime, so my coping mechanism in the meantime was to sit up in bed more, and to fit a black-out blind to the window.

This alleviated symptoms, but I knew I wasn't sleeping deeply, a short time later and a night of sleep monitoring it was confirmed, I have obstructive sleep apnoea.

I thought this was because I was fat, because I was getting older.  But it seems thats not just it, I am large, and all my family are.  However, the tiredness makes you want to eat more, the BBC has recently covered this phenomenon more than once, and I have to admit it was nice to hear.

I was issued with a CPAP machine, was immediately sleeping so much better.  It was like night and day.  And for the last year I have enjoyed this machine immensely.

However, the straps for the masks go slack over time, as they go slack your sleep falters again, as a man if you have too much facial hair air escapes the mask (have a beard but very short trimmed it's not a problem), far more of a problem is how the mask  seal skirt wear and then the straps go slowly slack.

I would have to argue that the company making these (Phillips) could be held somewhat to account for the latter, as it seems to happen at a predictable, almost designed rate, however I have to concede that the material used has certain requirements and the felt like vencro fastening straps with their slight elasticity are pretty good, but the continual re-stressing of the product does result in quite quick deterioration in performance, even before you take into account the biological factor (sweat & skin cells, and having to then wash the straps).

However a year on, I'm not wondering whether the machine is enough for my condition, I've noticed more than just slack straps, more than wearing seal skirts, more than just a bad nights sleep that more days than not I am quite tired of a day.

This is not everyday, but perhaps two days every fortnight or so, there's no obvious reason for this biologically, environmentally there's no changes either.  I have to admit to slightly more stress in my life at present, however pressure or stress have never stopped me sleeping before (a side effect of studying the martial arts - if a bloke bigger than me can hit me, a bit of code not working in the office is nothing to sweat over).

I will have to discuss this with my consultant in early June.

Being an every inquisitive soul I have started to look at other factors into why I might have apnoea whilst my brother (a close body match to me) does not, and as far as I know no-one else in my family - despite our having a string of heavy snorers - doesn't have apnoea.

One thing I mulled over, before doing some reading, is that during my mid to late teens I had orthodontic treatment, I had four healty pre-molar teeth removed and the two following molars behind these pulled forward.

I've just read, in two sources, that these molars play a role in strengthening the stiffening the sides of the soft pallet, and they also provide a framework onto which the throat muscles structure, then of course the net result is that your jaw is shrunk, leaving less room for the tongue.

I have to be completely honest here, if I let my tongue rest in as natural a position as it can, as easy feeling as it can be, it sticks out over the end of my lower front teeth.  But sits slightly behind my upper bite.

I have the distinct feeling that if I had not had this procedure, yes I would have slightly wonky front teeth, and yes there may not have been space for my wisdom teeth to fill in but my tongue would fit inside my teeth.

As a consequence of this poor fit I have to draw my tongue slightly backwards, with a kink mid-way, to let it sit behind my lower teeth, add the very nature of the action of the flesh being slack in my apnoea and this is a recipe for real trouble.

Having made mention of my weight problem, I did grow whilst without my CPAP machine, I went up to 21 stones around July 2016, by December I was 19 stones, and now today at posting, I am 17 stones and 9 pounds, you can go convert that yourself.

I'm still big, but I feel so much better, even with a few crappy days sleep I feel so much better.

Coming up in June, lets see if my consultant agrees with my orthodontic question, and if I need anything corrected or improved with my CPAP machine.

Wednesday, 10 May 2017

Development : Python, MySQL and Protocol Buffers Build

Today I've come to a totally virgin installation upon a server, this was for a work group I've got to head up whom are looking at pushing MySQL with Python.  And things initially went wrong...

I stipulated they had to use Python3 and thought everything else would be fine for them to install with Pip3, so...

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install python3
sudo apt-get install python3-pip
sudo apt-get install mysql-server

Everything looked fine, their user could install packages through Pip3 and they had Flask and a few other things and were flying.  I used the mysql client on the command like to test I could add a few items and also ran...

sudo mysql_secure_installation

To do the basic hardening of the database, so everything was fine... Right?....RIGHT?!?!

No, not exactly.... "I can't get mysql.connector".... Came the cry.

And they were right, it reported a myriad of build issues and could not install.  I took a look... NIGHTMARE!

It appears the installation of mysql.connector via Pip3 depends upon Protocol Buffers from google for the latest version of mysql.connector... Which the Pip install didn't sort out, at least not easily... Luckily I run a whole gaggle of virtualized machines, so I could quickly spool up a new instance and try a few things out...

This is the script I came up with....

cd ~
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install -y python3-pip git autoconf automake libtool curl make g++ unzip
sudo pip3 install flask
hostname -f > hostname.txt
sudo apt-get install -y mysql-server
sudo mysql_secure_installation
sudo ldconfig
cd ~
git clone http://github.com/google/protobuf.git
cd protobuf/
./autogen.sh
./configure
make
make check
sudo make install
sudo ldconfig
cd python
sudo python ./setup.py install
cd ~
sudo ldconfig
sudo pip3 install mysql-connector --install-option='--with-protobuf-include-dir=/usr/local/include/google/protobuf' --install-option='--with-protobuf-lib-dir=/usr/local/lib' --install-option='--with-protoc=protoc'

Lets go through this step by step...

cd ~
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install -y python3-pip git autoconf automake libtool curl make g++ unzip

This is just the basics, so our installation will now depend upon git, autoconf, automake, libtool, curl, make, g++ and unzip, most all your standard distributions will contain packages for these, we use -y just to skip any permission requests.

sudo pip3 install flask
hostname -f > hostname.txt

The next step is simply a couple of items for our project, we're going to use Flask to provide a restful interface, and the "hostname.txt" is simply to remove our need to call "hostname" again later.

sudo apt-get install -y mysql-server
sudo mysql_secure_installation

Our next step is to install and secure the MySQL service.

sudo ldconfig
cd ~

Generic code now, to simply reload the library list and change to the home directory.

git clone http://github.com/google/protobuf.git
cd protobuf/
./autogen.sh
./configure
make
make check
sudo make install

This is the build of protocol buffers from google, so we pull it from their github repo, we move into that folder, prepare and configure the build, then make the whole build.  By far this is the LOOOONGEST instruction, on a single core 1GB equipped virtual instance this took around 45 minutes.

Once complete we simply need to reload the libraries again...

sudo ldconfig

However, protocol buffers are still not installed within Python, so we are still in the "~/protobuf" folder, we now need to go deeper, into the python folder and perform the setup installation under python...

cd python
sudo python ./setup.py install

When complete we again need to reload the libraries...

sudo ldconfig

And the final, secret sauce, is to actually install mysql connector through pip3 with protocol buffers...

sudo pip3 install mysql-connector --install-option='--with-protobuf-include-dir=/usr/local/include/google/protobuf' --install-option='--with-protobuf-lib-dir=/usr/local/lib' --install-option='--with-protoc=protoc'

This is a single command, spanning one single line.

And voila, once complete you get to use mysql.connector in your python code...

import mysql.connector
import gc

con = mysql.connector.connect (user='whatever', password='something', host='localhost', database='yeahyeah')

con.close()
con = None

gc.collect()

You can find out more about why I nullify and garbage collect a connection in my previous post.