Burf.co is up!!! But….

So the good news is that Burf.co Search Engine is up and running in a rather alpha stage, the bad news it’s full of adult material so please be warned!!! There is no safe search filter on it yet!!!

My plan is to improve the search algorithm and then put a adult content blocker in place to protect people! This of course can be turned off but trust me, there is some nasty stuff out there that no one wants to see!

Technology wise, the index is just over a million pages and will heavily increase when the new blade server arrives tomorrow. It’s using MongoDB full text search for the initial results. MongoDB did seem to sh@t the bed when indexing all the words on a page, so I look at only parts of each page.

More to come soon ūüôā

This weeks update : Bye Bye MongoDB

So it is exciting times! I have made some progress with¬†TRTLExchange, however, due to things outside of my control it been slower than expected.¬† So I have turned my spare time to Burf.co, my new search engine project and while there is no website for it yet (will be by the weekend), the actual search technology(code) has come along leaps and bounds.¬† Overnight it managed to index over 500,000 pages which for a single server, was pretty cool.¬† It did get up to 1.3 million pages but MongoDB has erm, shit the bed(many many times).¬† This could be a hardware limit (Harddrive speed) or some performance thing I need to do however it gets to the point where I can’t even insert more records without timeouts.¬† This concerns me quite a bit as I have a HP Blade Server on way to somewhat up the crawling rate by a factor of 8.¬† I am going to try and give it one last go today however its taken 12 hours to delete the data from the DB (I did remove instead of drop ūüôĀ ).¬† It has been a very interesting learning curve on learning MongoDB.¬† I think unless some magic happens I am going to try out Postgres next.

On the Swift front I did start building the frontend for Burf, first I was going to do this in VueJS, however, I have now found that Swift’s server-side framework Perfect supports templating via¬†Mustache.¬† I think I will make faster progress writing it all in Swift than switching back and forth.¬† ¬†I still want to continue learning VueJS on the side (used for the TRTLExchange) as Javascript is such a good thing to know nowadays.

Writing this blog post has also just raised the point that I was trying to learn Kotlin about a month ago (facepalm).  Damn!


Experimenting with MongoKitten

As mentioned in my previous post, I have started looking at Server Side Swift with the aim to build a search engine (Burf.co).  To store my crawled data I decided to try and use MongoDB as it supports full-text search out of the box.  The original Burf.com used Equinox (made by Compsoft) and then later used Microsoft Indexing Service.  This time round I wanted to be a little more scalable.  Now there are probably better DB solutions for what I plan to do, but MongoDB seemed really simple to get up and running with.  Later on, I should be able to switch out the database layer if needed.


Now that I had decided to use Swift, and MongoDB, I needed to find a framework that connects them, my friend (who knows his stuff) recommended MongoKitten!  I got up and running with it fairly quickly even though I don’t know MongoDB too well. Life was good, however, there were a few things I did struggle with:


So, search a field for a partial string requires you to use Regex (it seems).  


db.users.findOne({“username” : {$regex : “.*eBay.*”}});


let query: Query = [

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†“url”: RegularExpression(pattern: “.\(eBay).”)


let matchingEntities: CollectionSlice<Document> = try pages.find(query)

Sorting results on $meta textScore

MongoDB allows you to setup full text searching across your data, it can be across an entire record, or just certain fields (name, address etc).  When you perform a full-text search, MongoDB returns the relevant records with an accuracy score ($meta.textScore).  MongoDB lets you change how it creates these scores by allowing you to adjust the weights each field receives e.g name is more important than address.


db.pages.find( {$text: {$search: “ebay”}},{score: {$meta: “textScore” }}).sort({score: {$meta:”textScore”}})


let query: Query = [

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†“$text”: [“$search”: str ],

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†“lang” : [“$eq”: “en”],


let projection: Projection = [

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†“_id”: .excluded,

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†“url”: “url”,

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†“title”: “title”,

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†“score”: [“$meta” : “textScore”]


let sort: Sort = [

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†“score”: .custom([

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†“$meta”: “textScore”



let matchingEntities: CollectionSlice<Document> = try pages.find(query, sortedBy: sort, projecting: projection, readConcern:nil,  collation:nil, skipping: 0, limitedTo: Settings.searchResultLimit )

Getting Help

I found the best way to get help was to contact the creator(Joannis) of MongoKitten via Slack, he is pretty busy but super helpful!

This weeks update : Server side Swift and updates to the site!

So, in terms of coding, the last week has been rather busy, actually in terms of progress, it’s looking good on all front (except Hack24)!

First off, there have been some updates to the website to finally include a basic list of historic Robotics projects.  I had planned to do that last year!  I have also updated the design and content a little, which frankly no one will notice!

Next, I started yet another new project, Burf Search Engine (The Return).¬† It had been on the cards for a while, however, I have now started coding it.¬† First in NodeJS, then in Server Side Swift using the Perfect framework.¬† It went swimmingly well (thanks to Ad) until I had to deploy it to Ubuntu.¬† There, you start to learn the differences between Apple frameworks and the open source versions of the same framework. Randomly I am expecting the Apple framework to have less //todo and //to implement comments ūüôā

The Crypto¬†project didn’t¬†process as far as I hoped this week purely¬†due to my friend (and other coder) being ill, however, momentium¬†has now picked back up.¬† As your also probably aware Cryptocurrencies took a massive dive this week!

No update on Hack24, it will resume after phase 1 of the¬†Crypto project has been completed and the search engine crawler for Burf.co is live ūüôā

On the side note, really enjoying Altered Carbon on Netflix’s and Thanks to Ford¬†for sending me this epic LEGO set 75885 (Ford Fiesta M-Sport WRC) to review at the weekend.



Humble Bundle: Mobile Development Bundle

So a new Humble Bundle has appeared that’s aimed at mobile devs which is cool!¬† It contains a large range of books by Packt which will help any Mobile Dev improve their skills, details below:


I am sure I have done it before, but I would also like to point out that Packt also gives away a free book every day!!!  Claiming it is one of my daily tasks!



Where is Hack24?

So Hack24 is becoming a bit more delayed than hoped, however, this is for good reasons (Well I think so).

I have been introduced to cryptocurrencies, via my friend sending me a link to https://turtlecoin.lol/ and telling me to mine the sh*t out of it!

Knowing a little (and I mean a little) about BitCoin (and its huge growth in value), I thought it was a great opportunity to find out more about this technology without investing any money.

Since I started mining it about 2 weeks ago, I have watched several documentaries with my wife about digital currencies plus the actual TRTL coin has gone up 100x in value!

If nothing else it’s been a fun learning experience and I feel a bit more comfortable¬†around the subject.¬† I am not working on a very short-term¬†project in this space ūüôā

Hack24 will resume very shortly ūüôā

It’s all go on Hack24

So, work is continuing on Hack24. Tonight I worked on making the actual buildings in the game. The colors are a bit naff but that won’t take much to sort out.

It was quite interesting to see the old Hack24 iOS codebase which was really nicely structured until you got to the building code (the code that generated the buildings in the game). Randomly the Y and Z axis had been switched over? No idea why!

There is still quite a lot to do but I am confident that Hack24 will be coming to the AppStore very soon!  After that, the website needs an update!  Finally, I hope to populate the Robots section on this site!

Week 7 and its server time!

So my mission was to release a basic version of Hack24 before Christmas, this is now seeming unlikely!¬† I do have a 24 hour GameJam coming up which I plan to abuse and work on Hack24.¬† I may just release the game on a single platform first (probably Android).¬† The main delay has actually been the Christmas spirit and wanting to do Christmas things ūüôā

So, back on to the progress. I have started building the user registration server in NodeJS. The original Hack24 used ASP.net which worked well but felt a bit of an overkill.  As I want to rapidly prototype the game, NodeJS seemed a good way to go.  Mongo is being used as the datastore and Socket.IO will handle real-time movement tracking.

For the MVP I need the following:

  • User can Register
  • User can Login
  • User can post high scores
  • User can see high scores
  • Users in-game¬†location is stored
  • Game world stored

Nice to have:

  • Reset password
  • User items
  • User Level
  • Admin¬† panel
  • User chat

In the original I used Apples GameCenter to do a lot of this, however as this is now multiplatform, I cant do this.¬† I also don’t really want to mix up Google Play users with GameCenter users and Steam users, it could all get a bit confusing.

Once this is done, I can then focus on finishing the game mechanics

We have Music, Sound, UI but no Dependency Injection

So another night coding Hack24 fueled by sugar and Redbull (well cheapo versions). I wanted to try and get Dagger2 working on my project so I could do Dependency Injection which is something I am interested in learning.¬† ¬†I timed boxed it to 1 hour and failed ūüôĀ This was due mainly to not understanding how it works with Kotlin. This project has a lot of moving parts to make my life more difficult, if it was a straight forward Android project, life would be fairly simple, however, it uses the LibGDX game engine which is cross-platform and normally in Java, and I am writing it in Kotlin. There seems to be not a single example anywhere of doing these 2 things together. My issue at the moment is with Gradle and the mass of gradle.build files that LibGDX has.

On the positive side, I did add a sound manager that does sound effects and music (quick win), I skinned the buttons, added the Hack24 logo and started building the factory methods to build the buildings in the game!

My focus is to build a minimal viable product of Hack24 before Christmas and release it on a single platform (Probably Android) as a soft launch.

Main areas I need to look at still:

  • Networking and the server side
  • UI for hacking a building
  • The player logon model, e.g Apple Game Center, Google Play etc