Insights

Development insights, client advice and news from Go Tripod

Everyone&#8217;s talking about <a href="https://openai.com/blog/chatgpt/">ChatGPT</a>, a language model optimised for conversations. In a shameless attempt to ride the gravy train, I asked ChatGPT for some help in writing a blog post.
<blockquote>Me: I need to write a blog post ASAP. Can you help?
ChatGPT: Sure, I&#8217;d be happy to help! What would you like the blog post to be about?
Me: Software. Software consulting, engineering, that kind of thing. And it needs to be stuffed with great keywords. Could you give me a sample and keep it under 50 words?
ChatGPT: Here&#8217;s a sample blog post about software consulting and engineering:
Are you looking for a software consulting and engineering firm that can help you achieve your business goals? Look no further! Our team of experienced software experts can help you design, develop, and deploy custom software solutions that are tailored to your specific needs. With a focus on agile methodologies and cutting-edge technologies, we can help you stay ahead of the competition and drive your business forward. Contact us today to learn more!
In this post, some key words and phrases that could help improve search engine optimization (SEO) include &#8220;software consulting,&#8221; &#8220;software engineering,&#8221; &#8220;custom software solutions,&#8221; &#8220;agile methodologies,&#8221; and &#8220;cutting-edge technologies.&#8221; Of course, the specific keywords that will be most effective will depend on your audience and your goals.
Me: That&#8217;s more than 50 words! Never mind, what hot new tech do you think we should talk about?
ChatGPT: I&#8217;m sorry, I didn&#8217;t realize that you wanted me to keep the post under 50 words. Here&#8217;s a revised version that is within the word limit:
Looking for a software consulting firm? Our team of experienced experts can help you design, develop, and deploy custom solutions. Contact us today!
As a large language model trained by OpenAI, I don&#8217;t have the ability to browse the internet or keep up with the latest trends in technology. However, some hot topics in the software industry currently include cloud computing, artificial intelligence, and machine learning. These technologies are being used to develop new software solutions that can improve efficiency, automate tasks, and provide insights that were previously impossible to obtain.</blockquote>
This comes across as a perfectly serviceable bunch of advice from someone familiar with software, SEO and copywriting. It might be lacking in spark, but let&#8217;s be honest, people that grind out SEO content aren&#8217;t going to win awards for creativity. I have a feeling we&#8217;ll see a lot more auto-generated content in the very near future. Want to see if you can use models like this in your project? <a href="https://gotripod.com/contact/">Give us a shout</a>.

Getting an AI to promote a software consultancy

Back in 2018 we talked about <a href="https://gotripod.com/insights/we-work-remotely-why-and-how/" target="_blank" rel="noreferrer noopener">how we make software development work remotely</a>, and about our tentative efforts to embrace <a href="https://gotripod.com/insights/a-caring-community/">Cornwall&#8217;s software development</a> community. Four years ago is a long time ago by any stretch, but in these difficult times it seems like an eternity. How have we at Go Tripod been affected and how have we changed?



<h2 id="h-us">Us</h2>



The biggest (non-software!) development has been our shift to a four day <a href="https://www.4dayweek.co.uk/">week</a>. For the past year we&#8217;ve trimmed the normal working week down by adding Friday on to the weekend, a 50% increase in free time. And there are no compromises &#8211; salary remains the same and the increased leisure time hasn&#8217;t come at a price.



Our clients have always had support requirements that mean we need cover outside of our own business hours, and this change hasn&#8217;t affected that. Someone is always on call to deal with urgent requests. 



We continue to work remotely, investing in equipment and people rather than office space. Slack is our main point of contact, and we use <a href="https://whereby.com/">Whereby</a> for conference calling.



<h2>You</h2>



We&#8217;ve recently been working with Unlocking Potential, an organisation based in the new <a href="https://fibrehub.uk/">Fibre Hub</a> down in Pool, to try and see how our services can better&#8230; well, serve, companies in Cornwall who are looking to undergo digital transformations. The use of software in this process is key, and we&#8217;re engaging with a number of small businesses who are developing cloud-based software to power the next phase of their business.



<h2>Us &amp; You</h2>



The past few years have shown many people what&#8217;s important in their lives, and we&#8217;re no different. We&#8217;re refocusing on our strengths, and remembering why we started working in Cornwall in the first place. If you&#8217;re looking for a partner to develop web-based software for your Cornish SME, you&#8217;re in the right place. We&#8217;ll help you develop your idea, explore funding options, and bring your software project to fruition. And if you just need some advice, we can act as your <abbr title="Chief Technical Officers">CTO</abbr> on an ad-hoc basis, so you have expert advice on-tap. Sound good? <a href="/contact/">Get in touch</a>!

On software development in Cornwall in 2022

We recently built a tool to visit every page in a website and collect the resulting data into a JSON file, a process known as web crawling and scraping. The JSON file was then used to create a search feature on the site, no amendments were needed to the code that powered the site itself. Here&#8217;s how we did it, and why we&#8217;ve open sourced the code to do it.



<h2 id="h-the-decision-making-process">The decision making process</h2>



There are a lot of solutions for crawling/spidering/scraping websites. A lot. And so obviously we decided to write our own so that it may become the <a href="https://xkcd.com/927/">universal standard</a>! Actually, no. We built what we needed to get the job done well using an open source <a href="http://go-colly.org/">scraping framework called Colly</a>. But why couldn&#8217;t we just use something that already existed, one of the many tools designed for this purpose?



Firstly, there is no one-size-fits-all solution for crawling. Every website is different. No matter which tool is used, you will have to specify what information you need, from which pages, and how you&#8217;d like it presented. Secondly, while we&#8217;re not adverse to using SAAS solutions, in this case we wanted to be able to control our costs and run the crawler on our schedule without worrying about spiraling costs. These two issues mean that we need an easily configurable tool that we can run without jumping through too many hoops.



We&#8217;d already had experience with scraping tools that could be configured with a GUI and so strongly favoured something which would let us capture our configuration as code &#8211; therefore allowing us to store it under source control.



<h2 id="h-colly-web-crawling-and-scraping-in-go">Colly &#8211; Web Crawling and Scraping in Go</h2>



After much research, Colly ticked our boxes: we could run it ourselves, we could configure it with code, and it was simple to set up and understand. Let&#8217;s take a look at an example, which is written in Go as you might expect:



<pre class="wp-block-code line-numbers"><code>c := colly.NewCollector(
 colly.AllowedDomains("gotripod.com"),
)

c.OnRequest(func(r *colly.Request) {
 fmt.Println("Visiting", r.URL.String())
})

c.Visit("https://gotripod.com/")</code></pre>



This has three distinct parts:



<ol><li>We create a &#8220;collector&#8221;. It is only allowed to visit gotripod.com</li><li>On each request, it should print the current URL</li><li>Visit the Go Tripod homepage, starting the site crawling process</li></ol>



All this will do is hit the GT homepage and stop, because we haven&#8217;t told it what to do afterwards. And since we&#8217;re writing a crawler and not a visitor, we need to tell it where to crawl to next. After creating the collector, we can add the following code:



<pre class="wp-block-code"><code>c.OnHTML("html", func(e *colly.HTMLElement) {
 e.ForEach("a&#91;href]", func(_ int, el *colly.HTMLElement) {
 e.Request.Visit(el.Attr("href"))
 })
}</code></pre>



When Colly&#8217;s collector detects that the requested URL is an HTML page, it&#8217;ll trigger this callback and loop through every <code>&lt;a&gt; </code>element with an <code>href</code> attribute. It&#8217;ll then visit the URL in the <code>href</code>. That&#8217;s really all we need to crawl through a website with Colly! This example code doesn&#8217;t do much, so let&#8217;s have a look at how we can scrape information from the page.



The <code>OnHTML</code> method takes two parameters, a selector and a callback. That selector is anything supported by <a href="https://github.com/PuerkitoBio/goquery">goquery</a> and if you&#8217;re comfortable with CSS selectors then you&#8217;ll be good. This selector specifies the element(s) that will be passed as the second argument to OnHTML, which has a type of <a href="https://pkg.go.dev/github.com/gocolly/colly#HTMLElement">colly.HTMLElement</a>. With the methods on that type you can further focus down your selection:



<pre class="wp-block-code"><code>c.OnHTML("article", func(e *colly.HTMLElement) {
 exportArticle(
 e.ChildText("news__meta"), e.ChildText("news__title")
 )
}</code></pre>



Every time Colly sees an article element on the current page it&#8217;ll pass it to the callback, where we use the ChildText method to select metadata and title for export. Colly has <a href="https://github.com/gocolly/colly/tree/master/_examples">loads of examples</a> which go into further depth, so we&#8217;ll leave our quick overview here.



<h2 id="h-all-aboard">All aboard!</h2>



Having been suitably impressed by Colly&#8217;s simplicity, we completed our project without fuss. It&#8217;s not the first time we&#8217;ve written a crawler and so we wanted to see if we could easily use Colly in future projects without rewriting the same code. We wondered if we could extend Colly&#8217;s use of selectors so that we could just reconfigure the crawler to run on a completely different project. It turned out we could!



Super Simple Scraper takes a JSON configuration file and pushes it into Colly&#8217;s crawl engine. Out pops a JSONL file. Let&#8217;s take a look at a configuration snippet. Web crawling and scraping ahoy!



<pre class="wp-block-code"><code>"html": {
 "selectors": {
 "title": "title",
 "h1": "h1",
 "url": "{{ .Request.URL }}"
 }
}</code></pre>



This is the configuration used when scraping HTML pages. We specify that there should be a selector called &#8220;title&#8221; which grabs the contents of the title tag. Likewise a selector called &#8220;h1&#8221; that grabs the combined contents of all the h1 tags on the page. The last selector is a templated selector which grabs the URL of the page. 



We&#8217;ve provided a Docker image to run SSScraper, which means that you can run it with a single command:



<pre class="wp-block-code"><code>docker run -it -v pwd:/go/src/app -e "CONFIG=$(cat ./path/to/your/config.json | jq -r tostring)" gotripod/ssscraper:main</code></pre>



<h2 id="h-ss-scraper-provides-results">SS Scraper provides results</h2>



And that&#8217;ll produce a JSONL file, where the selector names are used as the JSON keys:



<pre class="wp-block-code"><code>{"h1":"Want the internet to work for you? Got an idea for a project?","title":"Contact Go Tripod | website \u0026 web application development in Falmouth, Cornwall","url":"https://gotripod.com/contact/"}
{"h1":"Topic: Git Got an idea for a project?","title":"Git Archives - Go Tripod","url":"https://gotripod.com/topics/git/"}</code></pre>



That&#8217;s some Super Simple Scraping! The &#8220;title&#8221; and &#8220;h1&#8221; selectors are straightforward, but there&#8217;s some interesting stuff happening in the &#8220;url&#8221; selector. Let&#8217;s break that down to understand it a bit more.



We&#8217;re using Go&#8217;s text/templates package, and passing in the current request as a value. The selector {{ .Request.URL }} therefore becomes: &#8220;give me the value of the URL field on the Request variable&#8221;. 



Templates also have access to <a href="https://masterminds.github.io/sprig/">sprig</a> helpers, which means you can get fancy with something like:



<pre class="wp-block-code"><code>{{ .Request.URL.Path | regexFind \"\/(insights|work)\/\" | replace \"/\" \"\" }}</code></pre>



That would return either &#8220;insights&#8221;, &#8220;work&#8221;, or an empty string, depending on which appeared in the URL of the current page.



<figure class="wp-block-pullquote"><blockquote>Why JSONL? Because we can then import it straight into <a href="https://typesense.org/">Typesense</a> to search it!</blockquote></figure>



<h2>Custom software for custom requirements</h2>



SS Scraper doesn&#8217;t do anything that someone familiar with Go couldn&#8217;t do, but it does mean that we can quickly deploy a crawler/scraper on a completely new project without having to write code. We can have different config files for different projects rather than having to start afresh each time. <a href="https://github.com/gotripod/ssscraper/">SS Scraper is open source</a> on GitHub, and if you&#8217;d like us to help you deploy a crawler or scraper for your site, please <a href="https://gotripod.com/contact/">get in touch</a>.

Web Crawling and Scraping Made Super Simple

Automotive industry manufacturer <a href="https://gotripod.com/work/sanden/">Sanden</a> recently asked us to import a new set of data into <a href="https://sandenaftermarket.com/">their site</a>, which we built on WordPress several years ago. While we knew this was going to be a one-off custom WordPress data import, there were a few things to consider when deciding how to implement it.
<ul>
<li>The import would take a while to run, and we needed to see some kind of progress indicator</li>
<li>Because the import time will exceed the allowed request timeout, we&#8217;ll need to process the import in the background</li>
<li>If anything went wrong during the import we&#8217;d need a log</li>
<li>It&#8217;d be good to do a &#8220;dry run&#8221; to check for problems without writing to the database</li>
</ul>
This was one of those situations where WordPress is a little bit more of a hindrance than another system might be. Because the structure of the WordPress database is one-size-fits-all, it can be a bit tricky to do the kind of SQL INSERT you might do with other systems, and so we needed to jump through a few hoops. With the requirements asking for a dry run toggle, a progress indicator and a log display, it made sense to build a custom WordPress data import plugin which would show up in the WP admin and let us build a user interface. It could also use the WordPress authentication mechanism to restrict access to authorised users.
Rather than reinvent the wheel, we used <a href="https://actionscheduler.org/">Action Scheduler</a> as the backbone for our import processing. It also provided hooks to allow us to generate log files. It comes with its own dashboard to allow you to diagnose any issues, which we used during development of our plugin. Once that had been selected, the core of the import process was very simple &#8211; checking that records didn&#8217;t already exist and using the <a href="https://www.advancedcustomfields.com/">ACF</a> API to create them where missing.
As well as these benefits, building a custom WordPress plugin gave us a means to logically separate our code. We could package it up in a zip file for easy installation and updates, making testing a breeze.

Custom WordPress data import – solving business problems with plugins

&#8220;Software dependency management&#8221; just sounds so glam, doesn&#8217;t it? If we rephrase it to say &#8220;do less, get more&#8221; it starts to sparkle a little. Let&#8217;s take a look at this essential topic and some rules that Go Tripod follow in our <a href="https://gotripod.com/work/" target="_blank" rel="noreferrer noopener">projects</a>.



A &#8220;dependency&#8221; refers to a unit of code that the rest of your code relies on. The most common way of incorporating it is to use the package manager associated with your language of choice. And so; in Ruby you might add a <a href="https://rubygems.org/" target="_blank" rel="noreferrer noopener">gem</a> to your project, Rust uses <a href="https://crates.io/" target="_blank" rel="noreferrer noopener">crates,</a> JavaScript has <a href="https://npmjs.org" target="_blank" rel="noreferrer noopener">npm</a>. Easy peasy.



Except <a href="https://www.youtube.com/watch?v=7mAFiPVs3tM" target="_blank" rel="noreferrer noopener">it isn&#8217;t</a>. It&#8217;s telling that searching for the title of this post on Google turns up the Wikipedia page for &#8220;<a href="https://en.wikipedia.org/wiki/Dependency_hell" target="_blank" rel="noreferrer noopener">dependency hell</a>&#8220;. Adding dependencies is one thing but we&#8217;re talking software dependency management here, and that involves not only keeping them up to date, but making sure you actually need those dependencies you&#8217;re adding.



<h2>Rule 1 of software dependency management: do you need it?</h2>



Let&#8217;s talk about that last point first. Here we go, we&#8217;re adding a dependency:



<pre class="wp-block-code"><code>npm install stringconcat</code></pre>



In our theoretical JavaScript project, we&#8217;re running a command to add a dependency called &#8220;stringconcat&#8221; so that we can join strings together. But do we need that? In JavaScript, we certainly don&#8217;t:



<pre class="wp-block-code"><code>const joinedString = 'my' + 'string';</code></pre>



This is a contrived example, but back in 2016  <a href="https://qz.com/646467/how-one-programmer-broke-the-internet-by-deleting-a-tiny-piece-of-code/" target="_blank" rel="noreferrer noopener">a very simply library called left-pad was removed from NPM</a> and caused a great deal of consternation.



So before you run that command willy-nilly, think: do you need it? Or can you just write it yourself?



<h2>Rule 2: is it any good?</h2>



Before adding the dependency, you need to make sure if it&#8217;s of sufficient quality. This doesn&#8217;t just mean that it has the features you need, or that you need to do an in-depth source code analysis, but you do need to perform some due diligence:



How popular is your dependency? Are there more popular alternatives?



Popularity doesn&#8217;t guarantee quality, but it&#8217;s a good indicator that the community relies on it. That should mean that people are willing to maintain it.



How many maintainers does it have?



Is there only one person working on this code? What happens if they get really busy?



Are there lots of open issues or pull requests?



If the code&#8217;s on GitHub, take a look at the issues and pull requests. Does it seem like these are neglected, or is the project team responsive and present?



There&#8217;s no hard and fast rule for assessing the health of a project, but thinking along these lines should allow you to get a feel for whether code&#8217;s going to be well maintained.



<h2>Rule 3: do you still need it?</h2>



We might find that our code evolves. We might find that <a href="https://caniuse.com/" target="_blank" rel="noreferrer noopener">platforms evolve</a>. In <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/padStart" target="_blank" rel="noreferrer noopener">those cases</a> we need to keep on top of our dependencies to make sure that we&#8217;re not bringing anything in unnecessarily. Particularly in cases where native implementations become available, we should use that to get a speed boost for free.



<h2>Rule 4 of software dependency management: keep up to date!</h2>



Rules 1 &amp; 2 are mere preludes to the killer pain of dependency management: updates. Let&#8217;s look at an example using Ruby on Rails 6, which introduced Action Cable to integrate WebSockets into your application. You try and upgrade Rails as follows:



<pre class="wp-block-code"><code>bundle update rails</code></pre>



And see the dreaded:



<pre class="wp-block-code"><code>Bundler could not find compatible versions for gem "actionpack"
&lt;INSERT ENDLESS MESSAGES&gt;</code></pre>



This is the reality of dependency hell. A gem relies on actionpack 5.x, so when you try and upgrade actionpack to 6.x you can&#8217;t, because of that one gem. So, you update that gem, but you can&#8217;t, because either it hasn&#8217;t been updated for Rails 6 or it in turn depends on another gem which is tied to Rails 5. Repeat this all the way down the dependency tree or until you have removed all of your hair and nails, whatever comes first.



There is no foolproof way around this, but the easiest way is to follow the other rules: keep your dependencies limited to ones that you really need and ones which are well maintained. Secondly, don&#8217;t pin to some random broken git revision, and don&#8217;t use your own forks of code. You&#8217;re asking for trouble. Thirdly, make updating your dependencies routine. Do it regularly and you&#8217;ll avoid having to wade through dependency hell and migration instructions in changelogs. Not only will your hair and nails thank you, but your project will too.

Software dependency management – a primer

At Go Tripod, we&#8217;ve been in the business of software development for a long time. Although Go Tripod was officially formed in 2009, we&#8217;ve got blog posts going back to 2007 which describe the technologies we were interested in at the time. My <a href="https://colinramsay.co.uk">personal blog</a> goes back to 2003! Here&#8217;s a quick list of the things we&#8217;ve been into during that time:
<ul>
<li>ASP.NET (with MVC using Castle Project or SubSonic)</li>
<li>PHP &amp; WordPress</li>
<li>Obj-C</li>
<li>Adobe Flex &amp; Flash</li>
<li>Ruby &amp; Ruby on Rails</li>
<li>JavaScript (React, React Native, Ext JS &amp; Sencha Mobile)</li>
</ul>
That&#8217;s a whole host of disparate technologies. Can&#8217;t we just settle on one thing?
<h2>Backend Dealings</h2>
ASP.NET is still a thing today, just as it was when we wrote about it in 2007. But in 2007, the web was changing rapidly. ASP.NET was clunky, relying on something called ViewState, a giant blob of data that got transferred on every page request. While it enabled rapid application development, the result wasn&#8217;t always maintainable or conducive to quick loading. I was an <a href="https://colinramsay.co.uk/ruby%20on%20rails/2005/04/06/ruby-on-rails-a-dabble.html">early fan of Ruby on Rails</a> and its organised approach to development (known as MVC), and it seemed like a breath of fresh air compared to the complexity of ASP.NET. We still use Ruby on Rails for our backend systems today, 14 years after we first started researching it.
On the way to 2019 though, we continued evaluating new software. Microsoft got on the bandwagon with the mostly-excellent ASP.NET MVC framework, but the Rails community was cranking out loads of great libraries to support an excellent foundation. PHP did the same, with frameworks like CakePHP but, at the time it was released, PHP had a reputation for being a bit of a mess.
<h2>Front &amp; Centre</h2>
The same applies on the frontend, though to a lesser extent. JavaScript&#8217;s always been the <a href="https://en.wikipedia.org/wiki/Lingua_franca">lingua franca</a> of the web, powering rich interactions in the browser. But to get round some of its quirkiness, we had <a href="http://prototypejs.org/">PrototypeJS</a> in 2005, <a href="http://prototypejs.org/">MooTools</a> in 2006, and <a href="http://jquery.com/">jQuery</a> in 2007.
<blockquote>jQuery inspired me to create <a href="https://colinramsay.co.uk/css/projects/c%23/fizzler/2008/10/19/fizzler-a-css-selector-engine-for-c-sharp.html">Fizzler, a selector engine for C#</a> in 2008. It was improved and redesigned by <a href="http://www.raboof.com/projects/fizzler/">Atif Aziz</a>, and he still maintains it today.</blockquote>
A big one for me was the release of <a href="https://en.wikipedia.org/wiki/Ext_JS">Ext JS</a> in 2007/2008, and I went on to author and co-author <a href="https://www.packtpub.com/books/info/authors/colin-ramsay">several books</a> on this rich framework.
In 2013, <a href="https://reactjs.org/">ReactJS was released</a> and was as much of a game-changer for frontend development as Ruby on Rails had been for backend. We jumped on it, and its sibling for apps, <a href="https://facebook.github.io/react-native/">React Native</a>, to let us build complex user interfaces in a way that was maintainable and testable.
<h2>Why tho?</h2>
What makes us continually search for a <a href="https://en.wikipedia.org/wiki/Build_a_better_mousetrap,_and_the_world_will_beat_a_path_to_your_door">better mousetrap</a>? Looking at the bigger picture gives a clear reason: the difference in developer experience between jQuery in 2007 and React in 2019 is huge. A better developer experience means that we can provide an end product that is better tested and is more usable. The gap between ASP.NET and Ruby on Rails is a bit more muddled; we could use ASP.NET MVC now, which is a much better product. But Ruby on Rails wins out, because it&#8217;s simpler to use and has a huge lead in community-provided libraries to help us turn out top-notch projects.
<h2>But&#8230;</h2>
There&#8217;s an outlier here. Something that has remained constant, like a comforting blanket. WordPress was released in 2003 and remains incredibly popular, powering over half the world&#8217;s websites. It&#8217;s not perfect, but it&#8217;s developed in the open and has a massive support community. We&#8217;ve used it for years and hope to keep on doing so for years to come. The recent release of Gutenburg gives us the chance to let users build amazing page layouts with ease, and using partners such as WPEngine means we can keep things secure and backed up without any worries.
<h2>Conclusion</h2>
The world of software moves quickly, but the goals remain the same: make great experiences for our clients and their customers. The technology should fade away into the background, but we know it&#8217;s there, quietly powering and empowering.

Evolving our technologies and learning how to learn

<a href="https://content.gotripod.com/images/2019/01/photo-1531030874896-fdef6826f2f7.jpg" data-size="1350x900"><img decoding="async" loading="lazy" class="alignnone size-large wp-image-471" src="https://content.gotripod.com/images/2019/01/photo-1531030874896-fdef6826f2f7-1024x683.jpg" alt="" width="1024" height="683" srcset="https://content.gotripod.com/wp-content/uploads/2019/01/photo-1531030874896-fdef6826f2f7-1024x683.jpg 1024w, https://content.gotripod.com/wp-content/uploads/2019/01/photo-1531030874896-fdef6826f2f7-300x200.jpg 300w, https://content.gotripod.com/wp-content/uploads/2019/01/photo-1531030874896-fdef6826f2f7-768x512.jpg 768w, https://content.gotripod.com/wp-content/uploads/2019/01/photo-1531030874896-fdef6826f2f7.jpg 1350w" sizes="(max-width: 1024px) 100vw, 1024px" /></a>
Just interested in the code? <a href="#migrator-code">Cut to the chase</a>.
GitHub and GitLab are competing services providing source control products based on Git. Source control enables us to keep track of all of the code changes we make to a project so that we can roll them back in the event of a problem. Seeing the history of a project can also help understand why decisions were made, years after they happened.
<a href="https://blog.github.com/2019-01-07-new-year-new-github/">GitHub have just announced</a> that they&#8217;re providing free private repositories for individuals. This is great news for the community at large, but since we run an &#8220;Organisation&#8221; account on GitHub, it didn&#8217;t directly affect us. However, it did give us a chance to re-evaluate how we host our source code. We previously self-hosted GitLabCE, which is a fine piece of software. However, it had two drawbacks:
1. We&#8217;re hosting another server, which requires updating and auditing. 
2. There are some third-party services which only interact with GitHub and not GitLab
We decided to close our GitLab server and move to GitHub, and the first step of that is moving our repositories. We had 132 hosted on that server, so moving them one-by-one simply wasn&#8217;t practical. Fortunately, both GitLab and GitHub have APIs which we could use to migrate automatically.
We&#8217;re going to use a Ruby script to call the APIs. The gitlab and octokit gems wrap the GitLab and GitHub APIs respectively, so once we&#8217;ve installed those we need to configure them:
<pre><code class="language-ruby">Gitlab.configure do |config|
 config.endpoint = GL_ENDPOINT
 config.private_token = GL_PRIVATE_TOKEN
end

gh_client = Octokit::Client.new(:access_token =&gt; GH_PRIVATE_TOKEN)</code></pre>
Having done so we can fetch all of our GitLab projects:
<pre><code class="language-ruby">gl_projects = Gitlab.projects.auto_paginate</code></pre>
Auto-pagination means we can grab all of them at once. Now, we need to do something with each project:
<pre><code class="language-ruby">gl_projects.each do |gl_project|
# do something in here
end</code></pre>
Firstly, we specify the destination repository in GitHub:
<pre><code class="language-ruby">destination_repo = "#{GH_ORG_NAME}/#{gl_project.name}"</code></pre>
GH_ORG_NAME is a variable containing the organisation which will contain this repository, in our case &#8220;gotripod&#8221;. You&#8217;ll see where to specify this in the full source code later.
Before we can grab the repo from GitLab, we need to make sure we&#8217;re a member of its project. That&#8217;s straightforward:
<pre><code class="language-ruby">begin Gitlab.add_team_member(gl_project.id, 4, 40)
 puts "You've been successfully added as a maintainer of this project on GitLab."
rescue Gitlab::Error::Conflict =&gt; e
 puts "You are already a member of this project on GitLab."
end</code></pre>
And we also need to create the destination repo on GitHub:
<pre><code class="language-ruby">begin gh_client.create_repository(gl_project.name, organization: GH_ORG_NAME, private: true)
 puts "New repo created on GitHub."
rescue Octokit::UnprocessableEntity =&gt; e
 # If error everything else could fail, unless the error was that the repo already existed
 puts "Error creating repository on GitHub: #{e.message}"
end</code></pre>
Finally, we can trigger the import!
<pre><code class="language-ruby">gh_client.start_source_import(
 destination_repo,
 gl_authed_uri(gl_project),
 vcs: "git",
 accept: Octokit::Preview::PREVIEW_TYPES[:source_imports]
)</code></pre>
<code>gl_authed_uri</code> is a method defined as:
<pre><code class="language-ruby">def gl_authed_uri(gl_project)
 gl_repo_uri = URI.parse(gl_project.http_url_to_repo)

 "http://oauth2:#{GL_PRIVATE_TOKEN}@#{GL_SERVER}#{gl_repo_uri.path}"
end</code></pre>
If we tried to import from the GitLab repository URL without doing this, it would throw an error telling us we don&#8217;t have authorisation to access it. With this, method, we&#8217;re using our private GitLab token to gain access.
For extra bonus points, we can get a report of the progress of each import using Octopoller:
<pre><code class="language-ruby">Octopoller.poll(timeout: 15000) do
 result = gh_client.source_import_progress(destination_repo, accept: Octokit::Preview::PREVIEW_TYPES[:source_imports])

 print "r#{result.status_text}"

 if result.status_text == "Done"
 nil
 else
 :re_poll
 end 
end</code></pre>
This keeps checking the import until it returns a status of &#8220;Done&#8221;, at which point the import is complete! This makes the migration process a lot slower though, because it&#8217;ll wait until GitHub confirms that it&#8217;s done its work. If you comment out this block, then you can speed through all of the repos and let GitHub do the work in the background. It&#8217;ll email you when an import has completed.
We&#8217;ve made <a id="migrator-code" href="https://github.com/gotripod/gitlab-to-github-migrator" target="_blank" rel="noopener">the full source code</a> of this basic GitLab to GitHub migrator available on GitHub (unsurprisingly!). There&#8217;s some additional configuration code and a simple feature to allow resuming interrupted migrations. We don&#8217;t provide support, but you&#8217;re welcome to open issues and pull requests to collaborate on improving it. There are certainly a number of improvements that could be made (for example allowing an option to disable the progress report without having to comment it out), but we&#8217;ll leave that for another post.
Grab our <a href="https://github.com/gotripod/gitlab-to-github-migrator" target="_blank" rel="noopener">GitLab to GitHub migrator</a> now!

Migrate from GitLab to GitHub

<a href="https://content.gotripod.com/images/2019/01/photo-1519635694260-6af6fc200c89.jpg" data-size="1942x1310"><img decoding="async" loading="lazy" class="alignnone size-large wp-image-461" src="https://content.gotripod.com/images/2019/01/photo-1519635694260-6af6fc200c89-1024x691.jpg" alt="" width="1024" height="691" srcset="https://content.gotripod.com/wp-content/uploads/2019/01/photo-1519635694260-6af6fc200c89-1024x691.jpg 1024w, https://content.gotripod.com/wp-content/uploads/2019/01/photo-1519635694260-6af6fc200c89-300x202.jpg 300w, https://content.gotripod.com/wp-content/uploads/2019/01/photo-1519635694260-6af6fc200c89-768x518.jpg 768w, https://content.gotripod.com/wp-content/uploads/2019/01/photo-1519635694260-6af6fc200c89.jpg 1942w" sizes="(max-width: 1024px) 100vw, 1024px" /></a>
As 2018 hurtles away from us in the rear-view mirror, 2019 leaps upon us with its loving embrace. It&#8217;s time for us to look back, and forward, and take stock of where Go Tripod has been and where it&#8217;s going.
<h2>Double Down on Software</h2>
Last year we focused on the things we do well. We started off by <a href="https://gotripod.com/insights/a-new-look-for-go-tripod/">launching our new website</a> which saw us double our visits and increase our visitor count. We added a host of <a href="https://gotripod.com/work/">case studies</a> (with many more to come!) and, by becoming the <a href="https://gotripod.com/insights/jack-trades-master-one/">master of one trade</a>, we let those new visitors know exactly what we&#8217;re about. We made inroads into working with the larger <a href="https://gotripod.com/insights/a-caring-community/">Cornish software community</a>, something that was long coming, and we finally settled on a range of <a href="https://gotripod.com/insights/how-we-host/">hosting options that provide security</a> and peace of mind for our clients. We worked with WordPress a lot, improving <a href="https://gotripod.com/insights/ensuring-security-wordpress-websites/">security</a>, <a href="https://gotripod.com/insights/enqueue-defer-scripts-wordpress/">performance</a>, and our general <a href="https://gotripod.com/insights/managing-change-requests-within-the-development-process/">development process</a>.
<h2>Kick On</h2>
2018 felt like a year of consolidation, of certainty, and of improvement. In 2019, we&#8217;ll continue to review all aspects of our business, but we&#8217;ll be looking to do more research into techniques to take us to the next level. We&#8217;re working with a range of clients, new and old:
<ul>
<li><a href="https://www.keysafetv.com/">Keysafe Tenant Vetting</a> to revamp their platform for a changing marketplace</li>
<li><a href="http://www.aftermarket-sanden-europe.com">Sanden International</a> to provide a new product catalogue infrastructure</li>
<li><a href="http://creativeengland.co.uk/investment/epic">The EPIC fund</a> to work on digital health products for Cornish business</li>
</ul>
There are some exciting technologies coming to maturity that we&#8217;d like to leverage. <a href="https://firebase.google.com/">Google Firebase</a> has the <a href="https://firebase.google.com/products/firestore/">Firestore</a> real-time database in beta, allowing us to provide resilient offline-capable databases to our projects. Facebook continues to develop the React library, with exciting new features such as <a href="https://reactjs.org/docs/hooks-overview.html">Hooks</a> and <a href="https://github.com/facebook/react/issues/13206">Suspense</a> coming soon, and those will help us improve performance when we create complex user interfaces. WordPress has released their long-anticipated version 5.0, and we&#8217;ll be looking to see how the new <a href="https://wordpress.org/gutenberg/">Gutenberg</a> editor can improve the experience for our clients.
We&#8217;re also implementing some of the ideas raised during our <a href="https://gotripod.com/insights/how-we-host/">hosting research</a> and moving <a href="https://gotripod.com/work/tdk/">TDK-Lambda</a>&#8216;s infrastructure to Microsoft Azure. This will provide greater resilience for their database and application, as well as opening up exciting options for future development.
We&#8217;re really excited about 2019. If you&#8217;d like to hear what we could offer you in the year to come, please <a href="https://gotripod.com/contact/">get in touch</a>!
&nbsp;

A Happy Go Tripod New Year

As part of our drive to <a href="https://gotripod.com/insights/jack-trades-master-one/">double-down on software</a> and provide a great experience for our customers, we&#8217;ve been reviewing our hosting infrastructure. We recently talked about how our WordPress clients enjoy <a href="https://gotripod.com/insights/ensuring-security-wordpress-websites/">security and redundancy</a>, and we want to continue to provide that same great experience for the various web apps that we host. In this post, we&#8217;ll look at the three providers we&#8217;ve chosen to partner with in order to make this happen.
<h2>Heroku</h2>
Heroku is known as a &#8220;<a href="https://en.wikipedia.org/wiki/Platform_as_a_service">Platform as a Service</a>&#8221; or PAAS, which means that it provides infrastructure to run an application while abstracting things like operating system updates and configuration of the underlying system components. It integrates with source control to allow us to deploy versioned code (and roll it back in case of issues!), and the <a href="https://www.heroku.com/dynos">dyno</a> model lets us easily deal with high-traffic scenarios. Because Heroku is a cloud-based system, there&#8217;s no single point of failure, which means that if something does go wrong, chances are Heroku will automatically switch things behind the scenes so visitors never see an issue.
Heroku is built on Amazon&#8217;s AWS platform and, as such, it complies with accreditations such as PCI Level 1 and ISO 27001. This, on top of the huge number of Heroku addons and the excellent administration tooling, means that Heroku is our primary choice when we need to host Ruby on Rails or PHP applications.
<h2>Microsoft Azure</h2>
We built a bespoke ASP.NET content management system for <a href="https://gotripod.com/work/tdk/">one of our clients</a>, and since Heroku doesn&#8217;t formally support ASP.NET, we needed to look elsewhere. Azure is Microsoft&#8217;s answer to Amazon&#8217;s AWS and their <a href="https://docs.microsoft.com/en-us/azure/app-service/app-service-web-overview">Web Apps</a> provide a similar deployment model to that provided by Heroku. We can leverage Azure Storage to store uploaded documents in redundant fashion, and use Azure&#8217;s SQL Server support to deliver scalable and highly available databases. While Heroku is perhaps easier to use, Azure is more flexible and lets us work at a lower-level, and as such we&#8217;ll be moving our client&#8217;s infrastructure to Azure in 2019.
<h2>Netlify</h2>
For simpler sites which require little-to-no dynamic content, we turn to Netlify. It offers the same source control-based deployment model as Heroku and Azure, but it doesn&#8217;t support projects that are powered by a programming language or framework like the previously mentioned Rails, PHP or ASP.NET. Instead, its strength lies in making static sites as good as they possibly can be, with free TLS/SSL, caching support, and a redundant delivery network. If needs be, we can augment the static site with some simple dynamic features such as forms, login &amp; registration, and <a href="https://www.netlify.com/features/">custom functions</a>.
<h2>Host with Go Tripod</h2>
You may notice that when we talk about these providers, we&#8217;re reusing certain key terms such as versioning and redundancy. That&#8217;s because these are features that we think should be included as standard by any responsible hosting provider. By setting ourselves up for success, we can focus more on building software and less on maintaining infrastructure. If you&#8217;re looking for peace of mind when hosting your business-critical applications then <a href="https://gotripod.com/contact/">get in touch</a>.

How we host

We&#8217;ve recently been extending our reach into the local area by approaching organisations that help connect businesses together. The first of these is <a href="https://www.falmouth.co.uk/shopsandbusinesses/go-tripod/">Falmouth&#8217;s Business Improvement Distract (BID)</a>, who help boost the town by investing in marketing, business support, events and a variety of improvement mechanisms. The second is <a href="https://www.softwarecornwall.org/member/gotripod/">Software Cornwall</a>, who connect and support Cornwall’s digital techology community. These are the first two in a series of partnerships we&#8217;re entering into in order to better understand the needs of the local community.

A Caring Software Community

We recently posted about <a href="/insights/improved-font-loading/">the font loading solution we use on our websites</a>. Further to implementing this technique we noticed that, on this very site, some of the fonts weren&#8217;t quite rendering correctly.
Look at the examples below. The font rendering issues were slight but there was a definite problem with uneven character weighting in the Google fonts we were using (Noto Sans and Cabin).
It&#8217;s particularly noticeable on the crossbars of the capital Hs but you can also see that some of the ascenders are too bold. Capital Ss also show problems where their terminals on the cap line are bolder than their finials on the base line. There is a similar issue with the bold lower case C. Finally, the apices of the lower case Ts aren&#8217;t really rendering at all!
(Also note that the rendering issues are only obvious in the first paragraph of text and in the bold title. The second paragraph of text appears to be rendering correctly.)
<div id="attachment_443" style="width: 1034px" class="wp-caption alignnone"><a href="https://content.gotripod.com/images/2018/07/font-comparison.png" data-size="1920x636"><img aria-describedby="caption-attachment-443" decoding="async" loading="lazy" class="wp-image-443 size-large" src="https://content.gotripod.com/images/2018/07/font-comparison-1024x339.png" alt="Comparative screenshots demonstrating font rendering" width="1024" height="339" srcset="https://content.gotripod.com/wp-content/uploads/2018/07/font-comparison-1024x339.png 1024w, https://content.gotripod.com/wp-content/uploads/2018/07/font-comparison-300x99.png 300w, https://content.gotripod.com/wp-content/uploads/2018/07/font-comparison-768x254.png 768w, https://content.gotripod.com/wp-content/uploads/2018/07/font-comparison.png 1920w" sizes="(max-width: 1024px) 100vw, 1024px" /></a>Font rendering issues (left) vs. correct font display (right). Click or tap image to enlarge.</div>
Tying down what was causing this proved tricky for the following reasons:
<ul>
<li>The problems are only apparent on Windows machines</li>
<li>The problems are more apparent in IE than Chrome</li>
<li>The problems are only apparent with certain font families</li>
<li>The problems are only apparent at certain font size/weight combinations</li>
</ul>
So, to get to the bottom of this we needed to unpick our font loading solution&#8230; We use <a href="https://www.fontsquirrel.com/tools/webfont-generator" target="_blank" rel="noopener">Font Squirrel&#8217;s Webfont Generator</a> to generate compressed <abbr data-tooltip="Web Open Font Format">WOFF</abbr> and WOFF2 files from the original <abbr data-tooltip="TrueType Font">TTF</abbr>s.
The generator offers three modes: Basic, Optimal, Expert. We&#8217;d entrusted it to &#8216;Optimal&#8217; and it transpires that this was causing our problems.
Font Squirrel&#8217;s &#8216;Expert&#8217; mode allows you to adjust a whole host of settings. (See below for a subset of these.)
<div id="attachment_441" style="width: 685px" class="wp-caption alignnone"><img aria-describedby="caption-attachment-441" decoding="async" loading="lazy" class="wp-image-441 size-full" src="https://content.gotripod.com/images/2018/07/font-suirrel-generator-settings.gif" alt="Font Squirrel generator settings" width="675" height="374" />Font Squirrel webfont generator settings</div>
It turns out that the culprit in this instance is Font Squirrel&#8217;s Truetype Hinting algorithm. <a href="https://en.wikipedia.org/wiki/Font_hinting" target="_blank" rel="noopener">Font hinting</a> is a complicated matter, <a href="https://blog.typekit.com/2010/12/14/a-closer-look-at-truetype-hinting/" target="_blank" rel="noopener">the nuances of which</a> are beyond the scope of this post, however, suffice to say that automating the hinting process via the default &#8216;Font Squirrel&#8217; setting proved problematic.
<blockquote>When we set Truetype Hinting to &#8216;Keep Existing&#8217; or &#8216;TTFAutohint&#8217; our problems vanished.</blockquote>
It&#8217;s worth noting that we&#8217;ve not seen this issue on any other projects so this setting must only affect certain fonts negatively. However, as a bonus, this change also involves a file size saving for the variants of Noto Sans our website uses. See the comparison below:
<div id="attachment_442" style="width: 1034px" class="wp-caption alignnone"><a href="https://content.gotripod.com/images/2018/07/font-file-sizes.gif" data-size="1440x287"><img aria-describedby="caption-attachment-442" decoding="async" loading="lazy" class="wp-image-442 size-large" src="https://content.gotripod.com/images/2018/07/font-file-sizes-1024x204.gif" alt="Comparative font sizes showing that 'Keep Existing' hinting option results in ~10% saving " width="1024" height="204" srcset="https://content.gotripod.com/wp-content/uploads/2018/07/font-file-sizes-1024x204.gif 1024w, https://content.gotripod.com/wp-content/uploads/2018/07/font-file-sizes-300x60.gif 300w, https://content.gotripod.com/wp-content/uploads/2018/07/font-file-sizes-768x153.gif 768w" sizes="(max-width: 1024px) 100vw, 1024px" /></a>Noto Sans font file size comparisons with different hinting options (&#8216;Font Squirrel&#8217; on the left, &#8216;Keep Existing&#8217; on the right)</div>
As such we&#8217;ve now added a config file which includes this change (<code>"tt_instructor": "keep"</code>) to our boilerplate repo.
Moral of the story &#8211; leave font hinting to the font designer!

WOFF font rendering issue on Windows, fixed

As developers, we have a seemingly endless supply of tools at our disposal thanks to the hard work and generosity of the wider development community who, by and large, share their hard work in the hope of making their peers&#8217; lives that little bit easier.
At Go Tripod we&#8217;re always on the lookout for new ways to streamline our processes and improve the quality of our work. Task runners are a great example of this.
Utilising JavaScript, programs like <a href="https://gulpjs.com/" target="_blank" rel="noopener">Gulp</a> and <a href="https://gruntjs.com/" target="_blank" rel="noopener">Grunt</a> can automate all kinds of labour-intensive tasks including, for example:
<ul>
<li>compiling, concatenating and compressing assets (eg. images, <abbr data-tooltip="Cascading Style Sheets">CSS</abbr>, <abbr data-tooltip="JavaScript">JS</abbr>)</li>
<li>live browser reloading for speedy development and testing</li>
<li>auto-prefixing for optimal browser support</li>
</ul>
With these jobs running automatically in the background we&#8217;re able to focus entirely on the task at hand. This increases productivity and makes sure the code we&#8217;re writing is as performant and compatible as it can be.
Our current build tool of choice is a modified version of the excellent <a href="https://github.com/ahmadawais/WPGulp" target="_blank" rel="noopener">WPGulp</a> by WordPress aficionado, <a href="https://ahmadawais.com/" target="_blank" rel="noopener">Ahmad Awais</a>. It does most of the things we need out of the box but since we use <a href="https://www.upstatement.com/timber/" target="_blank" rel="noopener">Timber</a> to separate logic from styling I&#8217;ve amended it to watch <code>.twig</code> files as well as <code>.php</code> files.
<div id="attachment_434" style="width: 640px" class="wp-caption alignnone"><img aria-describedby="caption-attachment-434" decoding="async" loading="lazy" class="size-full wp-image-434" src="https://content.gotripod.com/images/2018/07/Screen-Shot-2018-07-17-at-09.29.26.png" alt="" width="630" height="411" srcset="https://content.gotripod.com/wp-content/uploads/2018/07/Screen-Shot-2018-07-17-at-09.29.26.png 630w, https://content.gotripod.com/wp-content/uploads/2018/07/Screen-Shot-2018-07-17-at-09.29.26-300x196.png 300w" sizes="(max-width: 630px) 100vw, 630px" />WPGulp compressing assets and watching for changes</div>
Along with a few other minor adjustments, the result is a fast and feature-packed workflow which allows us to build WordPress sites of the highest quality in record time. 💪

The best Gulp build tool for WordPress

Ten years ago, just before Go Tripod was formed, I was an independent contractor with a strong eye on my bottom line, so <a href="https://colinramsay.co.uk/personal/projects/php/freshbooks/plastiscenic/2008/11/27/freshbooks-invoice-widget.html" target="_blank" rel="noopener">I created a way to track my income</a> at-a-glance. This was extremely useful at the time, as it helped a young Colin make sure he had enough pennies at the end of each month, and we now use a much more advanced version at GT to make sure we&#8217;re bringing in some good dollar every year.
<h2>The business challenge</h2>
We&#8217;d like to be able to quickly get information on our current financial performance, including monthly tallies and current debtors and creditors. This data should be available in a frictionless manner &#8211; without needing to log in to accounting software, crunch numbers or run reports.
<h2>The solution (In a nutshell)</h2>
In my decade-old solution, I was using Freshbooks for accounting. At GT we now use Xero, a cloud-based accounting program that makes the whole process of invoicing, filing returns and tracking payments much easier. We worked out a way to pull our financial information out of Xero and have it posted into Slack, the chat service that our team uses each day. We wrote a chatbot, called Yus, which we can use to request vital information about how well Go Tripod is performing. Yus will also give us a breakdown of bills that we need to pay and invoices that are outstanding.
<h2>The solution (The software side)</h2>
We built Yus using a combination of <a href="https://botkit.ai/" target="_blank" rel="noopener">Botkit</a> and <a href="https://xeroapi.github.io/xero-node/" target="_blank" rel="noopener">xero-node</a>. It runs on <a href="https://www.heroku.com/home" target="_blank" rel="noopener">Heroku</a>. Here&#8217;s a sample of what it can do:
<div id="attachment_426" style="width: 680px" class="wp-caption alignnone"><a href="https://content.gotripod.com/images/2018/06/yus-1.gif" data-size="670x368"><img aria-describedby="caption-attachment-426" decoding="async" loading="lazy" class="wp-image-426 size-full" src="https://content.gotripod.com/images/2018/06/yus-1.gif" alt="Invoking '@yus invoicing this month' in Slack" width="670" height="368" /></a>Yus giving us invoicing figures</div>
We can send Yus simple phrases and it&#8217;ll respond accordingly. This time round it&#8217;s decided to exaggerate somewhat 🤑
<div id="attachment_427" style="width: 680px" class="wp-caption alignnone"><a href="https://content.gotripod.com/images/2018/06/yus-2.gif" data-size="670x368"><img aria-describedby="caption-attachment-427" decoding="async" loading="lazy" class="wp-image-427 size-full" src="https://content.gotripod.com/images/2018/06/yus-2.gif" alt="Invoking '@yus who owes us' in Slack" width="670" height="368" /></a>Yus shows us which clients have outstanding invoices over 30 days old</div>
In this case we get a list of clients who we might otherwise not have chased for payment until our bookkeeper flagged this up. Now, Yus can help us liaise with our clients to make sure invoices don&#8217;t slip through the cracks.
<h2>Conclusion</h2>
Transparency is key. Low friction transparency is even better. By using chatbot software we can listen to the heartbeat of Go Tripod and take immediate steps to perform course corrections. Fancy building some clever technology to improve your business? <a href="https://gotripod.com/contact/">Get in touch</a>.
<h2>A little word about Yus</h2>
The name &#8220;Yus&#8221; comes from two letters in the Cyrillic script: Little yus (Ѧ ѧ) and big yus (Ѫ ѫ). We use this character to represent a tiny tripod in our email footers.

Using chatbot software to improve transparency – introducing Yus

An early temptation when setting up a development shop is to diversify: to provide logo design, SEO assistance, print work, marketing. The ability to sell yourself to clients as a one stop shop is compelling, with the idea being that you can capture more work and provide a better experience for customers.
At Go Tripod, we don&#8217;t work that way.
We&#8217;re software developers and web experts. We like focus. By being up front and telling clients that we concentrate on what we do best, we can avoid spreading ourselves too thinly. The flip side of being a one stop shop is that there&#8217;s a lack of group knowledge about software best practices, such as using source control, automated testing, or keeping up to speed with the latest tools.
What happens when our clients do need other services? Then we hit our network. We can reach out to those who are using our approach: focusing on what they do best. That means that we can call upon the outstanding people in a particular field, bringing in specialists where needed. We&#8217;ll use Teamwork to bring everyone together to make sure the lines of communication remain clear, and in doing so can create projects that are greater than the sum of their parts.
We know code, and we know the web. <a href="https://gotripod.com/contact/">Let us show you</a>.

Jack of all trades? Or master of one?

We&#8217;re in the early days of planning out a web design-and-build project with a new client. A key part of this process is the creation of a site map.
<h3>The Site Map</h3>
In its simplest form, a site map is a pictorial representation of the pages required of a website. With smaller websites this list of pages is easy to define and it&#8217;s not much more of a stretch to group them into a sensible order.
With larger web projects this task can be much more involved and producing a sensible site map relies on a host of other disciplines (like information architecture and user experience design) but the requirements of the job at hand fall mostly into the former camp.
It&#8217;s worth demonstrating that a sensible site map doesn&#8217;t merely need to show a breakdown of a website&#8217;s pages. Instead it should also show how they fit together, where multiple pages might be grouped via dynamic data sources and what kind of elements might be common to every page.
The trick is fitting all of this information into a straight forward infographic that stakeholders can immediately understand (and preferably print).
Enter our case in point (click to expand):
<a href="https://content.gotripod.com/wp-content/uploads/2018/04/site-map.png" target="_blank" rel="noopener" data-size="1262x883"><img decoding="async" loading="lazy" class="alignnone wp-image-274 size-full" src="https://content.gotripod.com/images/2018/04/site-map.png" alt="Example website site map" width="1262" height="883" srcset="https://content.gotripod.com/wp-content/uploads/2018/04/site-map.png 1262w, https://content.gotripod.com/wp-content/uploads/2018/04/site-map-300x210.png 300w, https://content.gotripod.com/wp-content/uploads/2018/04/site-map-768x537.png 768w, https://content.gotripod.com/wp-content/uploads/2018/04/site-map-1024x716.png 1024w" sizes="(max-width: 1262px) 100vw, 1262px" /></a>
Note that I have used the homepage to define the site-wide header and footer information as well as the main navigation options. It is clear each navigation option represents a top tier page and it is immediately obvious which option includes additional features, pages or sub-sections.
(Strictly speaking the header and footer should be represented on every page, rather than the homepage alone, but to harness the power of an infographic one must expect some inferral ability on the part of our audience.)
<h3>The User Flow Diagram</h3>
User flow diagrams differ somewhat from site maps in so far as they attempt to demonstrate how a user interacts with a website rather than how the website&#8217;s contents are hierarchically broken down.
Naturally there is some crossover between these two approaches and again the complexity of a user flow diagram will depend on the complexity of the website (or process) concerned.
One way to think of these things is that whilst both diagrams might utilise boxes and arrows, the user flow diagram is more likely to take the shape of a flow chart.
With our current project, there was a loosely defined need for a restricted client area that would allow logged in users to access (mostly product based) information unavailable to regular visitors.
Whereas our client had been quite clear about what content was required of the front-end, defining this &#8216;restricted area&#8217; proved much more elusive. Instead we were given a long list of bullet point thoughts and ideas regarding things they might like.
As it happens, attempting to formalise these loosely defined requirements suits the user flow approach better than the site map approach.
And here is the case in point (click to expand):
<a href="https://content.gotripod.com/wp-content/uploads/2018/04/user-flow-diagram.png" target="_blank" rel="noopener" data-size="1262x883"><img decoding="async" loading="lazy" class="alignnone wp-image-275 size-full" src="https://content.gotripod.com/images/2018/04/user-flow-diagram.png" alt="Example website user flow diagram" width="1262" height="883" srcset="https://content.gotripod.com/wp-content/uploads/2018/04/user-flow-diagram.png 1262w, https://content.gotripod.com/wp-content/uploads/2018/04/user-flow-diagram-300x210.png 300w, https://content.gotripod.com/wp-content/uploads/2018/04/user-flow-diagram-768x537.png 768w, https://content.gotripod.com/wp-content/uploads/2018/04/user-flow-diagram-1024x716.png 1024w" sizes="(max-width: 1262px) 100vw, 1262px" /></a>
Note that this diagram demonstrates how a user might interact with the website and that the flow, in this instance, is shown from left to right.
Starting with the login process, the customer has a number of options; accessing important information available from their dashboard, amending their account details, or drilling down into specific product information.
Again, stakeholders can immediately make sense of what&#8217;s been proposed. This diagram should, after all, reflect a sensible interpretation of the client&#8217;s hastily compiled requirement list.
But also, importantly, it serves as an early conceptual blueprint that encourages client feedback. The flow diagram, once finalised, will additionally help formalise the development process.
In conclusion &#8211; a collection of carefully thought out infographics can serve as a powerful tool in terms of both defining a new project and improving an existing website or set of software processes.
&nbsp;

Project planning with site maps and user flow diagrams

Back in February <a href="https://gotripod.com/insights/improving-site-response-times-deferring-downloading-web-fonts/">I wrote about a font-loading technique I&#8217;d implemented on a new project</a>, but there was a problem with it &#8211; the script ran on every page load meaning the browser cache was never used to serve the fonts. There are many variations on the same theme with some using session cookies to determine if it&#8217;s a user&#8217;s first visit to the site, but they all come with their own quirks to overcome. After further research and testing I decided that, while not fully supported yet, the most effective and future-proof solution to this issue is not a JavaScript solution, but a CSS one.
<h2>Enter our old friend, <code>@font-face</code></h2>
The <code>@font-face</code> rule has been around as long as I can remember but went steadily out of favour as the popularity of <abbr data-tooltip="Flash Of Unstyled Text">FOUT</abbr> techniques grew. This is because its default behaviour results in a <abbr data-tooltip="Flash Of Invisible Text">FOIT</abbr> while the font is downloaded and with the rise of mobile devices on sluggish connections users can be left with a blank page for seconds or even minutes. This is no longer the case thanks to a new attribute, <code>font-display</code>.
<a href="https://caniuse.com/#search=font-display" target="_blank" rel="noopener">Browser support for <code>font-display</code> is growing</a> with the latest versions of Firefox, Chrome and Safari all playing ball. It degrades gracefully for all other browsers, falling back to its default behaviour and, as an added perk, we don&#8217;t need to worry about users with JavaScript disabled.
<h2>Optimising fonts</h2>
Here at Go Tripod we use <a href="https://fonts.google.com/" target="_blank" rel="noopener">Google Fonts</a> for most projects but in their default format (<abbr data-tooltip="TrueType Font">TTF</abbr>) file sizes can be much larger than is required. On slower connections this means the web fonts will take noticeably longer to swap with the system fonts and it&#8217;s best we do everything to deliver these assets to the user as quickly as possible. We use <a href="https://www.fontsquirrel.com/tools/webfont-generator" target="_blank" rel="noopener">Font Squirrel&#8217;s Webfont Generator</a> to convert our TTFs from Google into <abbr data-tooltip="Web Open Font Format">WOFF</abbr> and WOFF2 formats which often reduces their size by over 50% meaning the time it takes to download them is significantly decreased. This coupled with the <code>font-display: swap;</code> attribute means we&#8217;re providing most users with a superior experience which doesn&#8217;t rely on JavaScript, falls back gracefully and will soon be supported by all major browsers.

Improved font loading

Go Tripod is based at Tremough Innovation Centre in Penryn, Cornwall, but our team members work remotely from locations of their choosing. By giving our staff the freedom to create their own environment, they&#8217;re able to work smarter and in the way they prefer.
How do we make sure that everyone&#8217;s on the same page, providing our clients with the best service we can? We&#8217;ve got a suite of tools that help us coordinate and communicate.
<h2>Email</h2>
It may be seen as old hat but for most companies, including us, email is still a huge part of our operation. It&#8217;s often the first point of contact for new enquiries, and it&#8217;s an important mechanism for us to keep in contact with our clients.
<h2>Slack</h2>
We use <a href="https://www.slack.com">Slack</a> as our main means of communicating internally. It&#8217;s a chat system that allows us to have distinct channels for a variety of different topics and typically we have one for each project. This gives our discussions focus, and allows us to have a history of what&#8217;s been said (which is searchable for future reference). If we need to bring in a partner, such as a client or a contractor, then we can invite them to a channel as a guest so we can collaborate more easily.
<h2>Teamwork</h2>
As a project progresses, we track the outstanding items and the evolution of features using a project management tool called <a href="https://www.teamwork.com/project-management-software">Teamwork</a>. Our clients (<a href="https://gotripod.com/contact/">you could be one too!</a>) get their own login so they can participate in discussions, provide feedback and help us understand any issues. We can also track our time to invoice accurately, and we can see, at-a-glance, the status of any of our current projects.
<h2>Appear.in</h2>
No matter how fancy our chatroom and project management tools are, sometimes a phone call is the best way of getting a point across. We&#8217;ll often use <a href="https://www.appear.in">appear.in</a> to share a screen between the team to demonstrate a new feature or discuss an implementation, but we&#8217;ll also use it to let our team catch up. It supports webcams too so we can even look at each other&#8217;s ugly mugs!
<h2>Yodel.io</h2>
Since we spend so much time on Slack, we wondered if we could hook phone calls into it. It turns out we can with <a href="https://www.yodel.io/">Yodel.io</a>. When someone calls our office number, if no-one&#8217;s there to pick up, it&#8217;ll be routed into Slack where it can be answered in the browser with the press of a button. We can even redirect the call to the correct person&#8217;s mobile or send it to a customised voice message if we&#8217;re all unavailable.
By using this set of tools we can work more efficiently and deliver better value to our clients. Working remotely is becoming increasingly popular, and at Go Tripod we&#8217;ve been remote-working advocates since our formation in 2009. In fact, we think that working remotely enables us to provide a better service. Why not <a href="https://gotripod.com/contact">try us out</a>?

We work remotely – here’s why and here’s how

We&#8217;ve worked with various content management systems (<abbr>CMS</abbr>) over the years but <a href="https://wordpress.org/" target="_blank" rel="noopener">WordPress</a> is our favourite. Thanks to its powerful user interface and open source architecture, it&#8217;s the most popular <abbr>CMS</abbr> on the planet. (In fact <a href="https://w3techs.com/technologies/history_overview/content_management" target="_blank" rel="noopener">its market share</a> is 10 times the size of its closest rival!)
Whilst WordPress empowers millions of users to publish their own content, it also attracts hackers eager to find and exploit vulnerabilities in the WordPress system itself and (more often) in the myriad 3rd party WordPress plugins. As such, security issues are found on a regular basis and plugged just as regularly with frequent updates.
At Go Tripod, we thoroughly test any updates in a staging environment to ensure the new code doesn&#8217;t cause any problems before it&#8217;s pushed live to our clients&#8217; websites. We maintain the same care and attention throughout our development practice, investing time into researching the best tool for the job at hand.
So, to ensure our WordPress clients don&#8217;t need to worry about the security of their own websites, we have recently partnered with a dedicated WordPress hosting company.
This means our WordPress clients enjoy:
<ul>
<li>daily backups restored with a single click</li>
<li>automated <abbr data-tooltip="Secure Sockets Layer">SSL</abbr>/<abbr data-tooltip="Transport Layer Security">TLS</abbr> certificates via <a href="https://letsencrypt.org/" target="_blank" rel="noopener">Let&#8217;s Encrypt</a></li>
<li>unlimited development/staging environments</li>
<li>rapid environment updates via <a href="https://git-scm.com/" target="_blank" rel="noopener">Git</a></li>
<li>a built-in one-click <abbr data-tooltip="Content Delivery Network">CDN</abbr></li>
</ul>
Do you have a WordPress site? Is it secure? <a href="/contact/" rel="noopener">Get in touch</a> if you&#8217;d like to know more about what we can offer.