Lets kick it off again!

So one of the only plus points of not getting any sleep due to the effects of the extreme diet for this bodybuilding show is insomnia, my mind gets very creative and forces me to start kicking off new ideas, projects, missions et!

So, if you saw my last post, I said I was gonna finish Hack24, fix Burf.co and sort the garage!   So far, Burf.co is back up, but about to completely change, the garage is nearly finished being geared up as a robotics lab and Hack24 has not moved.  I do want to finish hack24 but I don’t want to rush it and I want to harness my energy on some crazy robotics ideas while my brain still works 🙂

So the plan v2!  Warning it’s a little bonkers, even for me!

Build a backend set of machine learning API’s that Burf.co, mobile devices, and my robots use to send and retrieve data.  The idea is I could send it a question, a command or an image and it does some magic and responds.

  • So for mobile devices, they would send images and text to speech, it would return ImageNet classification or answers to questions.
  • Burf.co would become more of a knowledge base system using NLP to feed into other systems.
  • There would also be a public facing chatbot which would hopefully learn off of all of this.  Planning a system POC using AIML to test the waters
  • This would all somehow be also brought together to add some usefulness to my future robotics projects (image classification, knowledge base, etc)

I brought some odd bits of hardware, upgraded the Burf.co server, brought some domains, and started rewriting Burf.co in Java.  I decided I want to try and use a common language and randomly Java seemed the best fit (client, server, mobile etc)

It’s gonna be a slow progress but I think its gonna be exciting.

Let’s kick it all off!

Sorry for the last few months there has been little update. I decided that before I got too old I should enter a bodybuilding contest, honestly seemed like a good idea at the time. Well it’s been the hardest 3 months of my life to be honest. First off it isn’t the cheapest thing to do (I have spent £500 on chicken alone), you have to be super disciplined (up at 5, 1 hour cardio, training even if ill), you become very moody, tired and even get insomnia (writing this at 3am) because your constantly hungry! However with less than 2 weeks to go, I am still chuffed I decided to do it. On a positive side, because you don’t sleep as much, your brain seems to be come very creative!!!

So what’s the plan Batman?

So, I have many plans, lots of ideas and have been researching lots of stuff but I need to do some house keeping first!

1) Release a v1 of Hack24 cross platform to prove the framework works.

2) Fix Burf.co Search Engine because I turned it off.

3) Finish the garage so that large projects are possible 🙂

All of the above are in progress and I hope to have them done very soon. Then it is full speed ahead for some cool robotics / machine learning project that I will discuss in my next post 🙂

Burf.co paused, Hack24 Resumed…..

So I been on holiday driving across America, it was great fun!  I was hoping to be inspired on what to do with Burf.co, what’s the business plan, the focus etc.  Sadly absolutely nothing came to mind!  Not a dime!  Technically I can now index a lot of records pretty fast!  I have enough hardware to heat up a street of houses (more than my ring-main could handle) however I still don’t know what the point is.  So for the moment, it’s being paused!

With that in mind, I thought I would carry on with Hack24 and see if I can get it out the door.  I am still using LibGDX however, I have to moved to Multi-OS Engine for iOS and I am going to see if I can use FireBase for the backend.

Hopefully, by the time I have an MVP built, I should then know what I am going to do with Burf.co

 

 

Tumbleweed and that damn CommonCrawl

I hate by starting a post apologising for not updating my blog so I won’t do that!

I have been a bit busy with the new job I started 3 weeks ago, so most of my side projects have been paused!  However, work on Burf.co has gone 2 steps forward, a couple to the left and then a couple steps backwards, this is largely due to the awesome site CommonCrawl.org having a huge part of the Internet crawled and open for anyone to use! They have petabytes of web data open for anyone to use and there are some really cool examples of how to use it, most involving a huge amount of cloud power!! I did ponder for quite a while how I would store so much data!!! I found an interesting Java project that scans the index of the CommonCrawl for interesting file types (https://github.com/centic9/CommonCrawlDocumentDownload).

I took this project, hacked it about a bit and changed it so that it would only return URLs that are mine type HTML and that had a response status of 200. This gave me around 50 million URLs to play with which all had file pointers to the actual web page data. Because this data is compressed, it’s far quicker to download them from the CommonCrawl than actually scrapping the website itself. CommonCrawl also follows the Robot.txt which is far more than I have ever done :). So far the end result is that I can get around 5 million pages of data a day (from my home internet) compared to around 500k on a good day!  That’s a pretty good increase!

This weeks update : Bye Bye MongoDB

So it is exciting times! I have made some progress with TRTLExchange, however, due to things outside of my control it been slower than expected.  So I have turned my spare time to Burf.co, my new search engine project and while there is no website for it yet (will be by the weekend), the actual search technology(code) has come along leaps and bounds.  Overnight it managed to index over 500,000 pages which for a single server, was pretty cool.  It did get up to 1.3 million pages but MongoDB has erm, shit the bed(many many times).  This could be a hardware limit (Harddrive speed) or some performance thing I need to do however it gets to the point where I can’t even insert more records without timeouts.  This concerns me quite a bit as I have a HP Blade Server on way to somewhat up the crawling rate by a factor of 8.  I am going to try and give it one last go today however its taken 12 hours to delete the data from the DB (I did remove instead of drop 🙁 ).  It has been a very interesting learning curve on learning MongoDB.  I think unless some magic happens I am going to try out Postgres next.

On the Swift front I did start building the frontend for Burf, first I was going to do this in VueJS, however, I have now found that Swift’s server-side framework Perfect supports templating via Mustache.  I think I will make faster progress writing it all in Swift than switching back and forth.   I still want to continue learning VueJS on the side (used for the TRTLExchange) as Javascript is such a good thing to know nowadays.

Writing this blog post has also just raised the point that I was trying to learn Kotlin about a month ago (facepalm).  Damn!

 

Humble Bundle: Mobile Development Bundle

So a new Humble Bundle has appeared that’s aimed at mobile devs which is cool!  It contains a large range of books by Packt which will help any Mobile Dev improve their skills, details below:

https://www.humblebundle.com/books/mobile-app-development-books

I am sure I have done it before, but I would also like to point out that Packt also gives away a free book every day!!!  Claiming it is one of my daily tasks!

https://www.packtpub.com/packt/offers/free-learning

 

Where is Hack24?

So Hack24 is becoming a bit more delayed than hoped, however, this is for good reasons (Well I think so).

I have been introduced to cryptocurrencies, via my friend sending me a link to https://turtlecoin.lol/ and telling me to mine the sh*t out of it!

Knowing a little (and I mean a little) about BitCoin (and its huge growth in value), I thought it was a great opportunity to find out more about this technology without investing any money.

Since I started mining it about 2 weeks ago, I have watched several documentaries with my wife about digital currencies plus the actual TRTL coin has gone up 100x in value!

If nothing else it’s been a fun learning experience and I feel a bit more comfortable around the subject.  I am not working on a very short-term project in this space 🙂

Hack24 will resume very shortly 🙂

Hack24 V2 started :)

So, it may end up being called something else but I have now started writing a new (well kinda) game.  The last couple of weeks I have been bug fixing and improving the LibGDX framework / demo app that I made a few years ago.  Code can be found here: https://github.com/burf2000/BurfEngine

Yes, it has a terrible name!  Anyway it was my attempt at a simple game like Minecraft where you can place and remove cubes.  I added things like chunking, gravity, and culling.  It also has a database and network layer, custom collision code and works on iOS, Android, and Desktop.  It has been converted to Kotlin, which I am really liking 🙂

Now, after a few long nights of fixing, that code is being parked and anyone can use it!  I now plan to rip it apart and use the best bits to form the engine for Hack24 (v2).  Made aim is to make a 3D game that has the same look and feel as Hack24 that’s cross-platform and easy to develop further.

I will focus on a MVP first, which should not take too long, I will try and rewrite the server in NodeJS (Well KotlinJS).

So why am I doing this?  A few reasons:

I really liked Hack24 but it had some performance issues and really needs rewriting to use OpenGL VBO / VAO.  It also makes sense to make it cross-platform while I am at it.  I don’t fancy doing it in Unity and really want to learn Kotlin for work (which I can for LibGDX).  It would be nice if people enjoy playing it too, so I hope to make it bigger with more content 🙂

Watch this space!

 

 

VEX Worlds 2017 – Robotics Competition

Sorry for the late post about VEX Worlds, I thought I would have more time after worlds to catch up with stuff, sadly (well not really), the kids have been mega active.  My eldest son played his first football tournament, had a holiday, lots of family stuff!

So, VEX Worlds, what an amazing experience, I went along for the VEX EDR side of the competition (this year it was split EDR / IQ) as I was showing off the EDR Tank.  Sadly I had to leave the US early as my son, Max was ill.  Still a very cool experience!

So, the EDR tank, well it performed really really well in remote control mode.  I mean the thing was fairly slow but must have covered MILES!  The batteries never died on me, nor did any motors!  I did kill a few Omniwheels, however, that’s to be expected.   Even though I left early, the EDR tank did not and so others drove it around.  I have not received it back yet to see how bad it is now, but I am sure it will be fine.

The autonomous side was a bit of a failure, to be honest, and looking back I had set up myself to fail and I will explain why.  The autonomous side was using ROS (Robotic operating system) which is an industry standard.  I was using a Neato Lidar system which is awesome however it only had a range of 5 meters and SLAM (simultaneous localization and mapping) to work out where I was, and where I needed to go via building up a map.  SLAM works by detecting features of the surrounding area to work out where it is.  When you’re in a hall that’s hundreds of meters wide with very little features, a sensor with a range of 5 meters is practically useless.  In the end, I just showed kids how it worked on my laptop using RVIZ.  If I had to do this properly I would need to invest in a proper LIDAR system with a much greater range.  Another aspect which makes this very hard is all the people moving around, how can SLAM pick up features if they are constantly moving!

Overall, the EDR tank was hugely popular, I gave tons of fist bumps, high fives, etc, people just thought it was cool, just a little slow.

Next year, if I did a vehicle again, I would have to make it a lot faster and forget about advance sensors etc!

Here are some videos of VEX World and the EDR TANK: