tag:x,2008-06-01:kind/xGo Free Range Blog2022-11-09T12:34:00+00:00tag:gofreerange.com,2022-11-09:/supporting-open-source-software2022-11-09T12:34:00+00:002022-11-09T12:34:00+00:00Supporting open-source softwarejames-mead<h1 id="supporting-open-source-software">Supporting open-source software</h1>
<p>All of us at Go Free Range know that our careers and even the existence of the co-operative itself owe a lot to <a href="https://en.wikipedia.org/wiki/Open-source_software">open-source software</a> and the community around it. In particular we’ve used the <a href="https://www.ruby-lang.org/">Ruby</a> programming language, the <a href="https://rubyonrails.org/">Ruby on Rails</a> web framework, library hosting by <a href="https://rubygems.org/">Rubygems</a>, and the <a href="https://bundler.io/">Bundler</a> package manager in many of our client and internal projects.</p>
<p>Members of <a href="http://gofreerange.com/">GFR</a> have authored a number of open-source projects, e.g. <a href="https://github.com/lazyatom/engines">rails engines</a>, <a href="https://github.com/exciting-io/printer">printer</a>, <a href="https://github.com/tomafro/recap">recap</a>, <a href="https://github.com/floehopper/edge-redirector">edge-redirector</a> and <a href="https://github.com/freerange/mocha">mocha</a>. We also support the open-source community by reporting issues and submitting pull requests. We <a href="https://github.com/freerange/site">work in the open</a> as much as possible and encourage <a href="https://github.com/alphagov/whitehall">our clients to do the same</a>.</p>
<p>I want to give my colleagues and GFR a big shout-out for being hugely supportive of my work on <a href="https://github.com/freerange/mocha">mocha</a>, the Ruby mocking library that I authored way back in 2006. At times, like many open-source maintainers, I’ve found this to be an onerous and thankless task. However, since mocha was used in Rails itself for many years and is still used in plenty of <a href="https://shopify.engineering/">long-lived codebases</a>, I still feel a responsibility to maintain it. I don’t think I would’ve managed to keep working on it without the support from GFR.</p>
<p>We’ve been reflecting on the debt we owe to the open-source community and investigating ways we could recognise this. It’s quite hard to work out how to do this effectively. In <a href="https://blog.sentry.io/2022/10/27/we-just-gave-260-028-dollars-to-open-source-maintainers/">this article</a> Chad Whitacre explains the difficulties Sentry had in donating to open-source projects. While projects like <a href="https://thanks.dev/">thanks.dev</a> & <a href="https://www.stackaid.us/">StackAid</a> are doing pioneering work in this area, they’re not really useful to us, because we’re not (yet!) a product company.</p>
<p>In the spirit of starting small, we’ve all decided to join <a href="https://rubytogether.org/">Ruby Together</a> as <a href="https://rubytogether.org/developers#plans">Developer members</a>. We’ll make regular monthly payments to support work on Rubygems, Bundler, and other projects. We see this as the next step in our journey to support and appreciate the world of open-source software.</p>
<p>Please get in touch at <a href="https://ruby.social/@freerange">@freerange@ruby.social</a> on Mastodon or <a href="https://twitter.com/freerange">@freerange</a> on Twitter if you have any suggestions on how else we could support open-source projects.</p>
tag:gofreerange.com,2022-11-01:/mocha-v2-release2022-11-01T14:10:00+00:002022-11-01T14:10:00+00:00Mocha v2 releasejames-mead<h1 id="mocha-v2-release">Mocha v2 release</h1>
<p>This major version bump of the Ruby mock object library, <a href="https://mocha.jamesmead.org/">Mocha</a>, includes some fairly significant changes. So I wanted to expand a bit on the <a href="https://github.com/freerange/mocha/blob/c5cf3249d9706f3470cbfcfd76e97b4bae87a3d0/RELEASE.md#200">release notes</a> and give some more detailed guidance on factors to consider when upgrading.</p>
<ul id="markdown-toc">
<li><a href="http://gofreerange.com/#strict-keyword-argument-matching" id="markdown-toc-strict-keyword-argument-matching">Strict keyword argument matching</a></li>
<li><a href="http://gofreerange.com/#removal-of-deprecated-functionality" id="markdown-toc-removal-of-deprecated-functionality">Removal of deprecated functionality</a></li>
<li><a href="http://gofreerange.com/#dropping-of-support-for-older-versions-of-ruby-minitest--test-unit" id="markdown-toc-dropping-of-support-for-older-versions-of-ruby-minitest--test-unit">Dropping of support for older versions of Ruby, minitest & test-unit</a></li>
<li><a href="http://gofreerange.com/#acknowledgements" id="markdown-toc-acknowledgements">Acknowledgements</a></li>
</ul>
<p><em>TL;DR: If you’re using a non-ancient version of Ruby, you’re using a non-ancient test library version, you’ve already upgraded to Mocha v1.16.0, and you’ve fixed all the Mocha deprecation warnings, then the worst that should happen when you upgrade is that you’ll see some new deprecation warnings!</em></p>
<h2 id="strict-keyword-argument-matching">Strict keyword argument matching</h2>
<p>Previously Mocha parameter matching always considered a positional <code>Hash</code> as exactly equivalent to a set of keyword arguments. However, in Ruby v3, positional arguments and keyword arguments have been separated and, in Ruby v2.7, behaviour that would be different in Ruby v3 is flagged by deprecation warnings. See <a href="https://www.ruby-lang.org/en/news/2019/12/12/separation-of-positional-and-keyword-arguments-in-ruby-3-0">this article</a> for more details on the separation of positional and keyword arguments in Ruby v3.</p>
<p>To address this a new configuration option (<a href="https://mocha.jamesmead.org/Mocha/Configuration.html#strict_keyword_argument_matching=-instance_method">Configuration#strict_keyword_argument_matching=</a>) has been introduced in Mocha v2. This option is available in Ruby v2.7 upwards.</p>
<p>In Mocha v2 the configuration option defaults to <code>false</code>, but in a future version of Mocha it will default to <code>true</code>. When the option is set to <code>true</code>, Mocha parameter matching considers a positional <code>Hash</code> and a set of keyword arguments as <em>different</em> even if their “keys” and “values” are exactly the same, i.e. the parameter matching is stricter and some invocations which previously matched may no longer match.</p>
<p>When the configuration option is set to <code>false</code>, parameter matching that would behave differently if the option were set to <code>true</code> is flagged by Mocha deprecation warnings. Once all these deprecation warnings are addressed, the configuration option can safely be set to <code>true</code>.</p>
<p>It’s important to address this issue, because otherwise you may end up with passing tests that give you a false sense of security. See the examples below.</p>
<h3 id="keyword-argument-syntax">Keyword argument syntax</h3>
<p>An area of possible confusion is the Ruby syntax that distinguishes between a positional <code>Hash</code> and a set of keyword arguments. In particular the use of <a href="https://en.wikipedia.org/wiki/Fat_comma">hash rockets</a> (“=>”) does <strong>NOT</strong> imply a positional <code>Hash</code>. Instead what matters is whether the “keys” and “values” are surrounded by braces (“{ … }”).</p>
<p>The following code defines a method that in Ruby v3 expects to be called with a single keyword argument. That method is then called four times, twice with the correct keyword argument and twice with a positional <code>Hash</code> including a key with the correct name. You might be surprised that the 2nd call (i.e. <code>foo(:bar => 1)</code>) is passing a keyword argument.</p>
<pre><code>def foo(bar:); p bar; end
# Method called with correct keyword argument
foo(bar: 1) # => 1
foo(:bar => 1) # => 1
# Method called with positional Hash
foo({ bar: 1 }) # => ArgumentError: wrong number of arguments (given 1, expected 0; required keyword: bar)
foo({ :bar => 1 }) # => ArgumentError: wrong number of arguments (given 1, expected 0; required keyword: bar)
</code></pre>
<h3 id="example-with-relaxed-matching">Example with relaxed matching</h3>
<p>The parameters in the expectation include a set of keyword arguments, but the parameters in the invocation include a positional <code>Hash</code>. With strict matching disabled, these parameters match the expectation and the test passes. However, when <code>Example#foo</code> is invoked in production code in Ruby v3 an <code>ArgumentError</code> is raised, i.e. the passing test does not highlight that <code>Example#foo</code> must be called with a set of keyword arguments.</p>
<pre><code>class Example
def foo(a, bar:); end
end
class ExampleTest < MiniTest::Test
def test_foo
example = Example.new
# The parameters in the expectation include a set of keyword arguments
example.expects(:foo).with('a', bar: 'b')
# The parameters in the invocation include a positional Hash
# These parameters match the expectation and the test passes
example.foo('a', { bar: 'b' })
end
end
example = Example.new
example.foo('a', { bar: 'b' }) # => ArgumentError in Ruby v3
</code></pre>
<p>Note, however, that a deprecation warning is displayed:</p>
<blockquote>
<p>Mocha deprecation warning at example_test.rb:NN:in `test_foo’: Expectation defined at example_test.rb:MM:in `test_foo’ expected keyword arguments (:bar => “b”), but received positional hash ({:bar => “b”}). These will stop matching when strict keyword argument matching is enabled. See the documentation for Mocha::Configuration#strict_keyword_argument_matching=.</p>
</blockquote>
<h3 id="example-with-strict-matching">Example with strict matching</h3>
<p>With strict matching enabled, the parameters no longer match the expectation and the test fails. This test failure highlights that <code>Example#foo</code> must be called with a set of keyword arguments.</p>
<pre><code>Mocha.configure do |c|
c.strict_keyword_argument_matching = true
end
class Example
def foo(a, bar:); end
end
class ExampleTest < MiniTest::Test
def test_foo
example = Example.new
# The parameters in the expectation include a set of keyword arguments
example.expects(:foo).with('a', bar: 'b')
# The parameters in the invocation include a positional Hash
# These parameters no longer match the expectation and the test fails
example.foo('a', { bar: 'b' })
end
end
# When Example#foo is invoked in production code:
example = Example.new
example.foo('a', { bar: 'b' }) # => ArgumentError in Ruby v3
</code></pre>
<h2 id="removal-of-deprecated-functionality">Removal of deprecated functionality</h2>
<p>A bunch of deprecated functionality has been removed in Mocha v2. As long as you’ve previously upgraded to Mocha v1.16.0 and fixed all the deprecation warnings you shouldn’t have any trouble.🤞</p>
<ul>
<li>It’s no longer possible to pass <a href="https://mocha.jamesmead.org/Mocha/API.html#mock-instance_method"><code>API#mock</code></a>, <a href="https://mocha.jamesmead.org/Mocha/API.html#stub-instance_method"><code>API#stub</code></a> or <a href="https://mocha.jamesmead.org/Mocha/API.html#stub_everything-instance_method"><code>API#stub_everything</code></a> a single symbol argument to create a mock object responding to a method named according to that symbol argument. Such an argument is used to name the mock object itself; any stubbed methods and return values should be setup by passing a <code>Hash</code> into these methods or by calling <a href="https://mocha.jamesmead.org/Mocha/Mock.html#expects-instance_method"><code>Mock#expects</code></a> or <a href="https://mocha.jamesmead.org/Mocha/Mock.html#stubs-instance_method"><code>Mock#stubs</code></a>.</li>
<li>If <a href="https://mocha.jamesmead.org/Mocha/Expectation.html#yields-instance_method"><code>Expectation#yields</code></a> or <a href="https://mocha.jamesmead.org/Mocha/Expectation.html#multiple_yields-instance_method"><code>Expectation#multiple_yields</code></a> have been used to specify that a stubbed method should yield then the stubbed method must be invoked with a block otherwise a <code>LocalJumpError</code> will be raised.</li>
<li>The <code>Configuration#reinstate_undocumented_behaviour_from_v1_9=</code> method has been removed. If you have addressed the deprecation warnings for <code>API#mock</code>, <code>API#stub</code>, <code>API#stub_everything</code>, <code>Expectation#yields</code> and <code>Expectation#multiple_yields</code> as explained above then this configuration option is redundant.</li>
<li>The <code>Configuration.allow</code>, <code>Configuration.warn</code> and <code>Configuration.prevent</code> methods have been removed. Use <a href="https://mocha.jamesmead.org/Mocha.html#configure-class_method"><code>Mocha.configure</code></a> and/or <a href="https://mocha.jamesmead.org/Mocha/Configuration.html#override-class_method"><code>Mocha::Configuration.override</code></a> instead.</li>
<li>The <code>mocha/setup.rb</code> mechanism has been removed. Use one of the <a href="https://mocha.jamesmead.org/index.html#installation">supported installation mechanisms</a> instead.</li>
<li>The <a href="https://rubyonrails.org/">Ruby on Rails</a> plugin mechanism has been removed. Use one of the <a href="https://mocha.jamesmead.org/index.html#installation">supported installation mechanisms</a> instead.</li>
<li>A <a href="https://mocha.jamesmead.org/Mocha/StubbingError.html"><code>StubbingError</code></a> is now raised when stubbed methods are invoked in a test other than the one in which they were defined. This is to avoid unintended interactions between tests and hence unexpected test failures. A test should clean up any state that it introduces.</li>
</ul>
<h2 id="dropping-of-support-for-older-versions-of-ruby-minitest--test-unit">Dropping of support for older versions of Ruby, minitest & test-unit</h2>
<p>Mocha v2 drops support for older versions of <a href="https://www.ruby-lang.org/">Ruby</a>, <a href="https://rubygems.org/gems/test-unit/">test-unit</a> and <a href="https://rubygems.org/gems/minitest/">minitest</a>.</p>
<p>More specifically Mocha v2 only supports:</p>
<ul>
<li>Ruby v2.0 and upwards. In particular Ruby v1.9 is no longer supported. Note that support for Ruby v1.9.3 ended on <a href="https://www.ruby-lang.org/en/news/2014/01/10/ruby-1-9-3-will-end-on-2015/">23 Feb 2015</a>.</li>
<li>Gem versions of test-unit from v2.5.1 (released on 05 Jul 2012) upwards. Versions of test-unit from the Ruby v1.8 standard library are no longer supported.</li>
<li>Versions of minitest from v3.3.0 (released on 27 Jul 2012) upwards.</li>
</ul>
<h2 id="acknowledgements">Acknowledgements</h2>
<p>Many thanks to the following:</p>
<ul>
<li><a href="https://wasabigeek.com/">Nick Koh</a> for all his hard work on the strict keyword argument matching functionality.</li>
<li><a href="https://po-ru.com/">Paul Battley</a> and <a href="http://hlame.com/">Murray Steele</a> for testing preview releases of Mocha v2 on sizeable codebases.</li>
<li><a href="https://www.chao-xian.co.uk/">Kelvin Gan</a>, <a href="https://ollie.treend.uk/">Ollie Treend</a>, <a href="https://dilwoarhussain.com/">Dilwoar Hussain</a>, and the rest of the <a href="https://gds.blog.gov.uk/">GDS</a> developer team for helping me test preview releases of Mocha v2 on the sizeable <a href="https://github.com/alphagov/whitehall">alphagov/whitehall</a> codebase.</li>
<li>My <a href="https://gofreerange.com/">Go Free Range</a> colleagues, <a href="https://blog.chrislowis.co.uk/">Chris Lowis</a> and <a href="https://gofreerange.com/people#chris-roos">Chris Roos</a>, for funding a lot of my time working on Mocha, for reviewing code & documentation and for just generally being very supportive.</li>
</ul>
<p><em>This article was <a rel="canonical" href="https://jamesmead.org/blog/2022-11-01-mocha-v2-release">originally published</a> on James’ blog.</em></p>
tag:gofreerange.com,2022-06-20:/week-7002022-06-20T17:00:00+01:002022-06-20T17:00:00+01:00Week 700chris-roos<h1 id="week-700">Week 700</h1>
<p>Exciting times! We’re starting a new contract tomorrow. It’ll see us through until the end of July and then both parties can work out whether it makes sense to continue. The three of us are going to be involved for the first two weeks and then <a href="http://gofreerange.com/chris-lowis">Chris</a> is going to move on to a couple of other projects that we’d previously agreed.</p>
<p>Finding project work has felt a bit different this year. We’ve had a few experiences of people being very enthusiastic about working with us up until the point we’re almost ready to sign contracts and then it all goes quiet, or we learn that we’re not all required or that the project length is much shorter than anticipated. I have no evidence to suggest this is different to previous years but it feels as though it might be.</p>
<p>I’ve continued to do bits and pieces for <a href="https://www.circularrevolution.org/">Circular Revolution</a>. Mostly researching no and low code software they might use to replace some of their manual processes.</p>
<p>Our switch from HSBC to <a href="https://www.starlingbank.com/">Starling</a> is almost complete. We’ve been with HSBC since the earliest days of GFR, and although we’ve never really been happy with them, we’ve never felt strongly enough about it to change. I was finally motivated to do something about it when I had to visit a branch to collect a new security dongle and then spend time on the phone activating it. We decided to switch to Starling, based partly on <a href="https://bank.green/banks/starling_bank">their green credentials</a>, and have been really impressed so far. The whole experience makes HSBC feel very antiquated.</p>
tag:gofreerange.com,2022-05-27:/week-6972022-05-27T15:19:15+01:002022-05-27T15:19:15+01:00Week 697chris-lowis<h1 id="week-697">Week 697</h1>
<p>We continued on the hunt for our next project this week. It feels like a few opportunities are getting closer, we’re now at the stage of working out start dates and waiting for budgets to be agreed. Fingers crossed we’ll have something more concrete after the long bank-holiday weekend.</p>
<p>We spent some more time in-person (at The Office Group in Waterloo) and remotely working out some changes to how we manage finances internally (see <a href="http://gofreerange.com/week-696">last week’s notes</a> for the background). I like the way we’ve managed the decision making around this, following a <a href="https://www.sociocracyforall.org/group-decision-making/">sociocracy</a>-style process. We created a proposal based on our discussions last week, and I “owned” it while Chris and James asked questions and raised concerns. We then refined the proposal together and had another round to see if we had concerns. We’ve decided its safe enough to try from June, so we’re going to give it a go and see how we get on. Hopefully we can share what we came up with soon too.</p>
<p>We spent half a day on our retainer agreement with Lollipop helping their team answer questions using data. Next week we’re planning to do a bit of pro-bono work for <a href="https://www.circularrevolution.org/">Circular Revolution</a> who run a zero-waste toilet collection service for London’s boaters. Although I’ll be on the beach in sunny Broadstairs.</p>
tag:gofreerange.com,2022-05-20:/week-6962022-05-20T14:00:00+01:002022-05-20T14:00:00+01:00Week 696chris-lowis<h1 id="week-696">Week 696</h1>
<p>We rented some office space at Waterloo on Monday morning so we could get together to talk about whether and how to restructure GFR so that it better suits our individual circumstances.</p>
<p>In the past, GFR finances were pretty simple. We all got paid the same salary and our task was to bring in enough income to pay that salary, our fixed costs and, hopefully, generate a bit of surplus for a rainy day, a <a href="https://mission-patch.com">side-project</a> or <a href="https://gofreerange.com/projects#gfr-video">two</a> and to allow us to donate cash or time to things we believed in.</p>
<p>Over the past few years for globally-obvious reasons, and because we all have child- and other caring responsibilities, our ability and availability to work for GFR has changed. We’re currently working on some changes that will, hopefully, reflect our individual circumstances in the money we take out of GFR but also make it easier to decide what each of us works on when.</p>
<p>We asked for some suggestions in one of our co-op network chat groups and received the response: “We’ve been struggling with this for 10 years!” so we know we’re not alone. Although it is a struggle, and it sometimes requires difficult conversations about money and the value of work, it feels good to me that we’re able to have these conversations and make changes as we see fit.</p>
<p>It’s not all been introspective though. This week we’ve done a little bit of work for <a href="https://www.lollipopai.com/">Lollipop</a>. We agreed a, new for us, retainer-style agreement to provide a small amount of ongoing support after our full-time engagement ended. It’s a bit challenging to context switch to a different project for a few hours a week, but it’s also good to keep in touch with the people we met there.</p>
<p>I’ve continued to do a small amount of consultancy work (this week mostly helping review hiring plans) for <a href="http://gofreerange.com/asc">Access Social Care</a> via the <a href="http://gofreerange.com/disrupt">Disrupt Foundation</a>. James wrote an <a href="http://gofreerange.com/blog-post">excellent blog post</a> off the back of automating one of our manual tasks. And we’ve spoken to a few potential clients about projects they have coming up. We (and our bank manager) hope to have some work lined up in the not too distant future. But as always, if you have a GFR-shaped hole in your organisation please <a href="http://gofreerange.com/get-in-touch">get in touch</a>.</p>
tag:gofreerange.com,2022-05-18:/how-to-backup-google-drive-to-s3-using-the-aws-cdk2022-05-18T17:46:00+00:002022-05-18T17:46:00+00:00How to backup Google Drive to S3 using the AWS CDKjames-mead<h1 id="how-to-backup-google-drive-to-s3-using-the-aws-cdk">How to backup Google Drive to S3 using the AWS CDK</h1>
<p><em>An AWS CDK project to schedule the backup of a shared folder in Google Drive to Amazon S3 using a scheduled Fargate task on Amazon ECS to run <code>rclone sync</code></em></p>
<p>Way back in <a href="https://www.urbandictionary.com/define.php?term=The+Before+Time">the Before Time</a>, I did some work to automate a couple of our recurring manual <a href="https://harmonia.io/">Harmonia</a> tasks. One of these was the task to back up our shared <a href="https://www.google.com/drive/">Google Drive</a> to <a href="https://aws.amazon.com/s3/">Amazon S3</a>. Prior to this we’d been running the <a href="https://rclone.org/commands/rclone_sync/"><code>rclone sync</code></a> command manually on one of our local machines. One significant downside of this was that we each needed to have a local copy of all the files (~17GB), so I was keen to come up with an automated solution running in the cloud.</p>
<p>Unlike with our <a href="http://gofreerange.com/automatic-backup-of-trello-boards-to-s3-using-aws-cdk">Trello backup</a>, it wasn’t obvious to me how we could split the work up into tasks short enough to run as <a href="https://aws.amazon.com/lambda/">AWS Lambda</a> functions. Also, although it was interesting from an educational point-of-view, I felt as if the orchestration/coordination complexities introduced by splitting up the Trello backup tasks had been overly cumbersome. So I decided to explore the idea of spinning up some compute to execute a script in one go much more like how a <a href="https://en.wikipedia.org/wiki/Cron">cron job</a> would run on a traditional server.</p>
<p>I was (and still am) enjoying using the <a href="https://aws.amazon.com/cdk/">AWS CDK</a> and so after a bit of research, I decided to use the <a href="https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs-patterns.ScheduledFargateTask.html"><code>ScheduledFargateTask</code></a> construct which is one of the higher-level patterns made available in the CDK. This construct meant that it was relatively straightfoward to spin up a container on <a href="https://aws.amazon.com/ecs/">Amazon Elastic Container Service (ECS)</a> at regular intervals and execute a shell script on that container.</p>
<h2 id="scheduled-fargate-task">Scheduled Fargate Task</h2>
<p>The task needed <strong>access to</strong> the internet, but there was no need for it to <strong>be accessible from</strong> the internet. I could’ve run it on a private subnet, but this would’ve meant I’d need either a NAT Gateway (expensive) or to run a NAT Instance on <a href="https://aws.amazon.com/ec2/">Amazon EC2</a> (maintenance/complexity overhead). Since the tasks only run for a few minutes every week I was willing to sacrifice the extra security provided by a private subnet in favour of a simpler/cheaper system where the tasks run on a public subnet.</p>
<p>However, at that point <code>ScheduledFargateTask</code> only ran if its VPC had a private subnet - if there was no private subnet available, an error was reported. So I decided to take the opportunity to contribute to the AWS CDK project and opened <a href="https://github.com/aws/aws-cdk/pull/6624">a pull request to allow ECS tasks to run on a public subnet</a> which was released in <a href="https://github.com/aws/aws-cdk/releases/tag/v1.29.0">v1.29.0</a>. I really enjoy contributing to open-source projects like this - it’s a really good way to get a deeper understanding of how it all works.</p>
<p>Having incorporated that change, I used <a href="https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.ContainerImage.html#static-fromwbrassetdirectory-props"><code>ecs.ContainerImage.fromAsset</code></a> to define the container image using <a href="https://github.com/freerange/google-drive-backup/blob/19a065b9bfebe8a7a4cbdc9f3739d628261d9f2c/local-image/Dockerfile">a local <code>Dockerfile</code></a>. This installs <code>rclone</code> on an Ubuntu base image and copies <a href="https://github.com/freerange/google-drive-backup/blob/ffc52080da5de7b780ba6b50352d0147ffad793e/local-image/home/backup.sh">a backup script</a> and associated <a href="https://github.com/freerange/google-drive-backup/blob/ffc52080da5de7b780ba6b50352d0147ffad793e/local-image/home/rclone.conf">rclone configuration</a> files into the home directory. This means you need the <a href="https://docs.docker.com/engine/reference/commandline/cli/">Docker CLI</a> available locally when you run <code>cdk deploy</code> so it can build the container image and push it up to <a href="https://aws.amazon.com/ecr/">Amazon Elastic Container Registry</a> ready for use by ECS.</p>
<p>It turned out that using the <code>rclone sync</code> command on a Google Drive folder containing so much data needs quite a bit of CPU and memory, but it was easy to increase this from the default of ¼vCPU & ½GB to <a href="https://github.com/freerange/google-drive-backup/blob/ffc52080da5de7b780ba6b50352d0147ffad793e/lib/google-drive-backup-stack.ts#L30-L31">4vCPU & 16GB</a> so that the command ran very quickly. Even though this is pretty beefy, given that it only runs for a few minutes once a week, the cost is negligible.</p>
<p>The task is scheduled using the <a href="https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-applicationautoscaling.CronOptions.html"><code>CronOptions</code></a> interface. Configuration is supplied to the container via environment variables using <a href="https://www.npmjs.com/package/dotenv">dotenv</a>. Credentials for Google Drive are supplied via <a href="https://aws.amazon.com/secrets-manager/">Secrets Manager</a>. Those for the S3 bucket are made available via the IAM role assigned to the ECS Task and used by <code>rclone</code> with the <a href="https://rclone.org/s3/#authentication"><code>env_auth</code></a> option set to <code>true</code>.</p>
<p>The task is monitored with the excellent <a href="https://healthchecks.io/">Healthchecks</a> service which we were already using for the Trello backup. This is effectively a <a href="https://en.wikipedia.org/wiki/Dead_man%27s_switch">dead man’s switch</a> which alerts us if the script doesn’t complete successfully at a given frequency and within a defined grace period.</p>
<h2 id="reflections">Reflections</h2>
<p>Two years on, I’m really happy how this turned out. Once I’d got the backup running successfully, we’ve only had one failure which was due to a recent change to the Google Drive API requiring a newer version of <code>rclone</code>. This meant I had to dive back into the code again to fix it, but I found it pretty easy to find my way around again partly because there’s not actually very much code!</p>
<p>The source code for the whole CDK project is <a href="https://github.com/freerange/google-drive-backup">available on GitHub</a>.</p>
<p><em>This article was <a rel="canonical" href="https://jamesmead.org/blog/2022-05-18-how-to-backup-google-drive-to-s3-using-the-aws-cdk">originally published</a> on James’ blog.</em></p>
tag:gofreerange.com,2022-05-13:/week-6952022-05-13T12:30:00+01:002022-05-13T12:30:00+01:00Week 695chris-roos<h1 id="week-695">Week 695</h1>
<p>We’re still here! Albeit without <a href="http://gofreerange.com/ben-griffiths">Ben</a> who moved on to pastures new at the end of 2021.</p>
<p>It’s nearly two years (95 weeks) since our last weeknotes back in July 2020. Since then we finished working with <a href="https://www.theaudienceagency.org/">The Audience Agency</a>, built <a href="https://codeclubworld.org/">Code Club World</a> with <a href="https://www.raspberrypi.org/">The Raspberry Pi Foundation</a>, and have done some shorter stints with <a href="https://juliesbicycle.com/">Julie’s Bicycle</a>, <a href="https://sh24.org.uk/">SH:24</a>, <a href="https://www.accesscharity.org.uk/">Access Social Care</a> and <a href="https://www.lollipopai.com/">Lollipop</a>. <a href="http://gofreerange.com/james-mead">James</a> has also mostly finished rewriting <a href="https://mission-patch.com/">Mission Patch</a> from <a href="https://rubyonrails.org/">Rails</a>/<a href="https://edgeguides.rubyonrails.org/webpacker.html">Webpacker</a>/<a href="https://elm-lang.org/">Elm</a> to <a href="https://kit.svelte.dev/">SvelteKit</a>.</p>
<p>We’re currently on the lookout for our next project so if you’ve got anything you think we can help with, or you have an idea you want to chat about, then please get in touch.</p>
<p>We’re getting back into the habit of regularly meeting in person having spent the majority of the past two years entirely remote. We’ve been meeting in cafes and pubs which is great for the social aspect but not so good for getting work done. We’re starting to explore meeting rooms but maybe some kind of flexible office might be useful again in future?</p>
<p>We’re currently trying to work out what it means to be working for GFR. This is a recurring theme throughout our history but the world we now live in, and our individual circumstances, mean that it feels a bit different this time round.</p>
<p>Right, that’ll do for now. See you in week 790, I guess.</p>
tag:gofreerange.com,2011-11-18:/week-1482011-11-18T15:43:00+01:002022-04-22T17:58:00+01:00Week 148jason-cale<h1 id="week-148">Week 148</h1>
<p>Hello lovelies.</p>
<h3 id="friendly-introduction">Friendly Introduction</h3>
<p>The chaos goddess has shined her luminous bosom in my direction this week. Thanks are partly due to the beautiful unfolding of the cosmos and partly due to <a href="https://github.com/freerange/harmonia/commit/d4f0925026456e3da07cd3ca7107cf411516c9a3">this commit</a> in which <a href="http://interblah.net/">James A</a> managed to locate and unravel a kinky-kink in the space time continuum. Imagine a giant pearl necklace, no, even bigger than that, massive, each pearl the size of a million enchanted forests, but this is just a scale, it is much bigger, more real AND this is the tip of the iceberg of beauty in which we are swimming, imagine it.</p>
<p>Now come back.</p>
<p>So with our friendly introductory discourse dealt with, lets get straight to the hard facts.</p>
<h3 id="hard-facts">Hard facts</h3>
<p>We’ve been continuing with our work on Gov.uk, rolling out some bad ass features as standard. We switched to one week iterations from two week ones we started out with (this was initially to line up team working on the Citizen focused offering). I much prefer this as it feels like you can bury your nose in and focus, for a nice maintainable pace over the week, with a planning game before the weekend break, and we can come back fresh to it.</p>
<p>I guess tighter feedback loops with our users is also argument for doing it, but as we were deploying in shorter intervals than once per sprint anyway, we’ve been pretty tight with our users from the start.</p>
<h3 id="drawing-perfect-circles-for-infinity">Drawing perfect circles for infinity</h3>
<p>When we were working on <a href="http://hashblue.com">#blue</a> I was bothering <a href="http://tomafro.net/">Tom</a> about some link-to-and-back-from pages in the app dependent on the context of where the user has come from or was intending to do.</p>
<p>As usual Tom had “some code to do that” which we popped right in (after writing some tests for it, natch).</p>
<p>It turned out that we’d like something similar for Gov.uk, so we have made a Gem aptly called <a href="https://github.com/freerange/boomerang-rails">Boomerang</a> to wrap up the functionality. Our limbs cleansed and free, we can use it wherever we please — and because it’s <a href="https://github.com/freerange/boomerang-rails">on github</a> you can too!</p>
<h3 id="talkin-bout-the-smiling-geohash-blues">Talkin’ ‘bout The Smiling Geohash Blues</h3>
<p>Longtime readers (for at least a couple of weeks) may know about Tom’s <a href="http://tomafro.net/2011/09/a-small-toy-to-explore-geohashes">Geohash Toy Thing</a> with which you can get all crazy with the Geohashes. If you don’t know why you would want to go all crazy with the Geohashes, then you need to go listen to Tom talk about why you want to go all crazy with the Geohashes at the <a href="http://lanyrd.com/2011/geomob-november/">GeoMob November Shindig</a> happening next week.</p>
<h3 id="chris-wants-to-bag-it-up">Chris Wants To Bag It Up</h3>
<p>On tuesday our friends at Hackney Council came round and sorted out our recycling for us; for a little fee (bout a-quid-fiddy per bag) they will take away our paper mache monsters and turn them into fancy loo roll and organic hamster cardigans.</p>
<p>We are very happy about this because we like to keep our relationship with Mother Nature the <a href="http://0to255.com/16901e">same colour as our test suite</a>.</p>
<h3 id="party-karamu-fiesta-forever">Party, Karamu, Fiesta, Forever!</h3>
<p>Yesterday the INTERNET was ah-blaze with the news of our “totally rad” <a href="http://lanyrd.com/2011/go-free-ranges-office-warmpocalypse/">Office Warmpocalypse</a>. It was well Bieber.</p>
<p>Yes! Our massive space needs to be filled with toxic substances and human bodies in order to test the parameters of our fire code.</p>
<p>You, Dear Reader are forever invited to come and make history at the biennial Go Free Range office ceremony where we will all join hands and witness the consummation.</p>
<p>All the details are <a href="http://gofreerange.com/officewarming">laid out for your eyes</a> and it would be royally splendid if you would grace our humble abode with your beautiful instance of human.</p>
<h3 id="roos-is-a-lightweight">Roos is a lightweight</h3>
<p>Not sure what happened but our intrepid <a href="http://chrisroos.co.uk">hairy-code deleting-surf hippy-stalwart of good times-Chris-MotherFlippin-Roos</a> came in at like 2pm today. Rumour is going round that now that Chris has entered deep into Old Age (30) that he can no longer handle his drinks.</p>
<p><img alt="Chris: HairFACE" style="width:100%" src="http://gofreerange.com/assets/blog/chris-hair-face-6a81dfca1852d4411d8216dec0192829200bd7e74f6911f7203c458be2a22642.jpg" /></p>
<p>Better make it a shandy tonight then, Chris.</p>
<h3 id="dramatic-farewell">Dramatic Farewell</h3>
<p>I hope your mind is free and your hearts pumping fast for the weekend ahead.</p>
<p>May all your Oscar Wildest dreams come true.</p>
<p>JASE.</p>
tag:gofreerange.com,2020-07-17:/week-6002020-07-17T18:03:07+01:002020-07-17T18:03:07+01:00Weeks 586 to 600chris-roos<h1 id="weeks-586-to-600">Weeks 586 to 600</h1>
<p>From week commencing Mon 6 April to week commencing Mon 13 July.</p>
<ul>
<li>
<p>I can’t quite believe that it’s been about three months since I last wrote any weeknotes! <a href="http://gofreerange.com/chris-lowis">Chris</a> captured this period quite well for me when he described it by saying that the days have felt long but the weeks have flown by.</p>
</li>
<li>
<p>We ended <a href="http://gofreerange.com/week-585">week 585</a> with Chris being furloughed, <a href="http://gofreerange.com/ben-griffiths">Ben</a> and <a href="http://gofreerange.com/james-mead">James</a> starting work on <a href="https://www.futurelearn.com/">FutureLearn</a> and me working about half time on Lightbox. Our rough plan was for Chris to be furloughed for a short(ish) period, Ben and I to work about half time and for James to work roughly full time. But it didn’t quite work like that: it soon became clear that it wasn’t practical for either Ben or Chris to work with the kids at home and so we ended up furloughing them from early April until early June.</p>
</li>
<li>
<p>I managed to work about 3 days a week by sharing childcare with my partner and working some weekends. Our daughter went back to nursery a couple of weeks ago and so I’m now able to work 4 days a week, and have weekends back. I’ve spent the last three months working on Lightbox, the <a href="https://businessresponsecovid.org.uk/">National Business Response Network</a> and with <a href="https://www.theaudienceagency.org/">The Audience Agency</a>.</p>
</li>
<li>
<p>We resumed working with TAA in week 591. We’ve added artform coding to the app (replacing a very manual process), spiked on some potential Insights around Recency, Frequency, Value (using <a href="https://github.com/ankane/blazer">Blazer</a> to prototype), and started working out to provide more timely access to survey responses. I’ve mostly been pairing with James Trinder. We’ve been using <a href="https://tuple.app/">Tuple</a> for remote pairing which seems to work really well.</p>
</li>
<li>
<p>James has been working at least full time (and almost certainly more) over the last 3 months: working at least 4 days a week with FutureLearn and then a day or more on GFR related business. At FutureLearn, James has been involved in delivering COVID-19 related iterations of the <a href="https://www.futurelearn.com/info/research-insights/introducing-futurelearn-campus">FutureLearn Campus</a> and <a href="https://www.futurelearn.com/courses/collections/futurelearn-schools">FutureLearn for Schools</a> products. I’m very happy that he’s recently taken a week off but am also very conscious that things haven’t been equitable. I don’t know what we’ll do about it but hope we can do something to rebalance things in the near future.</p>
</li>
<li>
<p><a href="https://outlandish.com/">Outlandish</a> were good enough to think of us when they needed an extra pair of hands on the National Business Response Network project. I enjoyed the few days I had to work on the project. It’s written in <a href="https://laravel.com/">Laravel</a> which felt similar enough to Rails to allow me to feel fairly productive despite not having used it before.</p>
</li>
<li>
<p>Ben and Chris returned from furlough in week 595.</p>
</li>
<li>
<p>Ben contributed to another grant application with Manuel (product owner for Lightbox), did some work with <a href="https://wearecast.org.uk/">CAST</a>/<a href="https://www.thecatalyst.org.uk/">Catalyst</a> on a project aimed at helping charities, and has now started working with Outlandish on another of their projects.</p>
</li>
<li>
<p>Chris is mostly working on the TAA project with me but is also finding time to participate in a a <a href="https://www.thenews.coop/127746/topic/education/can-co-op-co-op-training-help-drive-worker-co-operative-development/">Barefoot Co-op Development Training course</a> and work on some fairly gnarly colour calculations in Lightbox. We’re making slow but steady progress on Lightbox and really appreciate Manuel’s patience while things have been so unsettled.</p>
</li>
<li>
<p>Ben, Chris and James collaborated on the application for a <a href="https://www.digitalmarketplace.service.gov.uk/digital-outcomes-and-specialists/opportunities">DOS</a> opportunity to upgrade Ruby and Rails on various <a href="https://gds.blog.gov.uk/">GDS</a> projects. We were shortlisted but unfortunately ended up just missing out to <a href="https://unboxed.co/">Unboxed</a>.</p>
</li>
<li>
<p>James has set-up <a href="https://sendboard.com/">Sendboard</a> which allows us to use <a href="https://trello.com">Trello</a> as a shared inbox. I appreciate not receiving all the group emails in my inbox but don’t feel that I’ve fully got used to the new workflow just yet.</p>
</li>
<li>
<p>I’ve been impressed by the furlough scheme. <a href="https://www.freeagent.com/">FreeAgent</a> generates a report with all the required information, the application form is very straightforward and the money appeared in our account within a week or so! It’s certainly helped to have some additional income during these tough months.</p>
</li>
<li>
<p>In contrast to the furlough process I found myself quite confused by the terminology/concepts in the <a href="https://www.gov.uk/guidance/check-if-youre-eligible-for-the-coronavirus-small-business-grant-fund">Small Business Grant Scheme</a>. We eventually got our application in and have now received the funds, which is a great relief.</p>
</li>
</ul>
<p>I’m sure I’ve missed lots of what’s happened but this should give a bit of a flavour of our life over the past few months.</p>
<p>Until next time.</p>
<p>– Chris</p>
tag:gofreerange.com,2020-04-07:/week-5852020-04-07T13:00:27+01:002020-04-07T13:00:27+01:00Week 585chris-roos<h1 id="week-585">Week 585</h1>
<p>Week beginning Mon 30 March.</p>
<ul>
<li>
<p><a href="http://gofreerange.com/chris-lowis">Chris</a> and his partner welcomed their second child into the world. I understand they’re all at home and well. Congratulations. This is wonderful news :-)</p>
</li>
<li>
<p><a href="http://gofreerange.com/ben-griffiths">Ben</a> and <a href="http://gofreerange.com/james-mead">James</a> started working with <a href="https://www.futurelearn.com/">FutureLearn</a> after they got in touch to ask whether we could help cover some of their team working reduced hours.</p>
</li>
<li>
<p>I worked on a new interface for the Lightbox project that aims to guide researchers through the process of including lighting data in their papers.</p>
</li>
<li>
<p>We agreed to exercise the break clause in our office lease only to learn that our landlord was happy for us to move to a rolling monthly contract at a greatly reduced rate to store our equipment. We’ve taken this option as it avoids us having to collect/store our equipment and gives us some hope we’ll be back in there in future.</p>
</li>
<li>
<p>We’ve explored some of the options available to help us through this time. We’ve furloughed Chris, postponed our next VAT payment, investigated the <a href="https://www.gov.uk/government/publications/guidance-to-employers-and-businesses-about-covid-19/covid-19-support-for-businesses#support-for-businesses-through-the-coronavirus-business-interruption-loan-scheme">Business Interruption Loan Scheme</a>, learnt that we’re hopefully eligible for the <a href="https://www.gov.uk/government/publications/guidance-to-employers-and-businesses-about-covid-19/covid-19-support-for-businesses#support-for-businesses-that-pay-little-or-no-business-rates">Small Business Grant Scheme</a> and have started exploring how much we can afford to reduce our salary by.</p>
</li>
<li>
<p>We’ve retired our self-hosted <a href="https://errbit.com/">errbit</a> instance in favour of <a href="https://rollbar.com/">Rollbar</a>. The cost of maintaining our own instance was outweighing any benefits we might’ve seen.</p>
</li>
<li>
<p>We’ve configured <a href="https://freerange.coop">freerange.coop</a> to redirect to <a href="https://gofreerange.com/">gofreerange.com</a>. We’ve had freerange.coop since becoming a member of <a href="https://www.uk.coop/">Coops UK</a> and figured we should do <em>something</em> with it even if that’s just configuring a redirect. We had a brief conversation about making it our primary domain but it felt like that warranted some more thought/discussion.</p>
</li>
<li>
<p>James has written a blog post to describe <a href="https://jamesmead.org/blog/2020-03-30-automatic-backup-of-trello-boards-to-s3-using-aws-cdk">how he’s using the AWS CDK to automatically backup our company Trello boards to AWS S3</a></p>
</li>
<li>
<p>We used <a href="https://whereby.com/">Whereby</a> for our monthly drinks on Wednesday. I thought it worked really well and I enjoyed spending an hour or two catching up with folks.</p>
</li>
<li>
<p>There’s a new Slack channel for the <a href="https://medium.com/wethecatalysts/catalyst-digital-teams-available-to-help-charities-respond-to-covid-19-b69b9e124b29">Catalyst Digital Teams network</a>, which alerted me to a <a href="https://outlandish.com/blog/products/24-hour-turnaround-covid-busting-business-broker-build/">project Outlandish completed for Business in the Community</a> in just over 24 hours! It’s inspiring to see people from various, potentially competing, organisations sharing leads and working out how to work together.</p>
</li>
<li>
<p>We used our monthly charitable donation to contribute to the crowdfunder for <a href="https://www.crowdfunder.co.uk/masks4nhsheroes">Personal Protective Equipment (PPE) for NHS Staff</a>.</p>
</li>
</ul>