Galaxy Admin Training with the Gallantries

The Gallantries is currently planning our first event: “the GTN Smörgåsbord: A Global Galaxy Course” and as of today the registrations have grown to 600 people from all across the world! This is an incredibly exciting development for our first official Gallantries event, but it’s also a bit terrifying as none of us have run such a big event before.

As such the Gallantries participated in the organisation of the 2021 Online Galaxy Admin Training, where we deployed the same methodology we intended to use during the Smörgåsbord! This gave us a much smaller and more reasonable test platform for evaluating the tools and methodologies we plan to use. Running a global course for 88 participants required us to develop a large number of innovations we’re very excited to share with everyone.

While previous events hosted by Galaxyproject have reached wide audiences within a timezone, this event has provided an ideal test case for a world wide training event, something not successfully achieved within our project before.

The Past

GAT has been a yearly event offered by the Galaxyproject. The main goal of GAT is for beginning Galaxy admins to gain the skills they need to run reliable production infrastructure. Every iteration of the event has involved major overhauls every time it has been taught. In the past 2 years we have finally settled on a standardised and reusable curriculum with only minor changes necessary each year.

When the COVID-19 pandemic struck at the beginning of 2020, we had literally just finished giving the last GAT. Most of the instructors flew home from Barcelona and began quarantining. When BCC2020 was moved online we knew we had to improve the situation for remote admin trainings. We developed new scalable, flexible instrumentation to monitor student progress and interact with them. We found that it worked acceptably, but BCC2020 called for synchronous interaction which proved extremely stressful for the instructors and students if they were in Africa, the Middle East, or Europe, and outside of the target timezones.

When GAT2021 would be held online we decided it was time to completely rethink how we give training. With some of the instructors from the Gallantries we identified a workable plan for an world-wide GAT.

The Present

Methodology

With a GAT pilot during the BCC2020 event, we taught a few of the lessons online and had some relevant experience:

• Materials must be online
• We need to be able to see student progress
• Synchronicity does not scale

For this event we decided to go fully asynchronous. We would place everything online and students could follow along at their own pace, taking breaks whenever they liked. We identified this as a maximally productive solution: we can reach the widest audience with the very limited number of qualified instructors.

Logistics & Communications

The workshop was well publicised across Twitter, the Galaxyproject websites, within ELIXIR and EOSC-Life, recruiting a large number of participants.

Our students were quite varied:

All participants received a pre-event email with basic instructions. The were directed to join the Slack we setup as the communication platform during the event.

We additionally had a FAQ document which we had used heavily in the past. The students would collect questions in the FAQ and we would answer them. After the event we would take every question asked in the FAQ and integrate it into the training material in some way.

However we found that to be ineffective in this specific context, as these materials have gone through numerous rounds of improvements. The students had few questions because everything was so functional, and when they did have questions, it was specific to their mistake, not something general covered by a FAQ.

We will try it again during the Smörgåsbord event where we expect it to be more effective.

Videos

The main, massive change with this event was the asynchronous nature. We created videos for as many topics as we could in advance (thank you EOSC-Life for funding this!) and then placed all of these on YouTube for our students. We identified subtitles early on as a key need for this event given the world-wide nature and number of non-native English speakers that would attend. Many of our instructors also have strong accents and we wanted the videos to be maximally accessible to our students. This is something we cannot achieve in real life that we can do better in an asynchronous event.

Video Production

Our instructors largely used Zoom for recording themselves doing the trainings which was quite effective. Zoom produces very small output files (~100 Mb for an hour of video) which significantly simplified sharing these and collecting them for upload. Once uploaded to YouTube, automatic captions were generated over the course of 8-24 hours.

These captions were copied to a public Google Document where we invited members of the community to volunteer to caption videos in exchange for recognition on the video.

YouTube has a nice feature where the entire contents of the video’s captions can be edited, and they will take care of syncing the audio to the correct location for you. Thus we could edit the captions separate from their timings and this was very easy for volunteers to listen to a video and correct spellings and sentence ends.

Subtitles	Plays	Watch time (hours)
Total	1579	248.3096
off	987 (62%)	148.573 (59%)
en	591 (37%)	100.6836 (40%)

By watch time 40% of our users watched with subtitles. In the preparation to this event we saw pushback on captioning, casting it as a waste of time. These numbers definitely show the important of captioning for our content.

YouTube

We had alternative video hosting platforms but considered that some students would have unreliable internet and YouTube has a significant POP facilities across the world that we could leverage for free. An added bonus was that YouTube provided a large number of statistics for us.

Schedule

We presented the students with a reduced schedule which we thought could be accomplished within the allotted time. We planned that the students should spend around 5 hours a day working on the material, intending this to be very flexible for them and fit with their schedules and lives. Here is a portion of the schedule:

Start today by introducing yourself in the #general channel in Slack! Tell us where you’re from, and one thing about your surroundings (e.g. it’s snowing outside, there’s a squirrel on my porch, my cat is on my keyboard)

Topic	Video	Slides / Materials	Slack Channel
Welcome and introduction	Video		#general
Intro to Ansible	Video	Slides	#01-ansible
Ansible	Video	Tutorial	#01-ansible
Galaxy Ansible Introduction	Video	Slides	#02-ansible-galaxy
Galaxy Installation with Ansible	Video[1]	Tutorial	#02-ansible-galaxy
Touch Base Telco (See Slack)

see the full schedule

Something the instructors noted in previous online events was that our bosses did not treat us as “busy”, as they would if we were physically in a location. A few of our students echoed this in their feedback. As a result we aimed for 5 hours of content, if you were reading the training material and working through it as you read.

Slack

The ice breakers turned out to be quite effective for getting students to interact and say hi. It let them share something and others comment on it, fostering discussion in what is otherwise a very isolating experience.

We intended to regularly poll students with a Slack message, but this turned out to not be necessary. The students were very active every day, and many voluntarily checked in with issues or just comments on the materials. This is probably going to be a more useful thing to do in the future Smörgåsbord event.

Lastly, we had chosen Slack as the platform due to the built-in ability for 1:1 video calls but these were unused as we could login to the student machines ourselves, and our students were technically savvy enough to share screenshots.

Helpers

The logistics of organising helpers across such a wide arrange of timezones was not a simple task! We started by breaking the world into three regions, APAC (or APO in our documents), EMEA, and the Americas. We recruited helpers from each of these regions, and had them report their availability during the week. But with this information we still didn’t have a good idea about potential gaps in coverage, so Helena Rasche of the Gallantries developed a novel, open source visualisation, marrying a timezone map of the world with a heat map

This was published and is free for anyone to use to visualise their own global events.

Participants

We monitored the progress of participants over the course of the week using a script which checked which tasks the students had completed on the VM allocated to them.

This gave us the ability to track progress without directly bothering or asking students, and gave us valuable data on their accomplishments.

Judging by the following graph, we may have overestimated the amount of content students could accomplish with the competing demands of their jobs (versus how bosses respond to someone being physically on a workshop trip.)

Feedback

The feedback from students was largely very positive. Most students found the training extremely useful to them.

Importantly for us, most students compared this event favourably with a face-to-face training, reporting “I am very happy with my virtual experience”.

Likewise, contrary to our expectations, the trainees reported a significant affinity for the videos.

We asked students about the best and worst aspects of the event. They identified a number of things they really liked

“Best” Aspect	People reporting this
You go at your own pace	7
Flexibility of schedule	4
Flexibility of location	3
Slack	3
Materials + Videos	2
Instructor responsiveness	1
[specific instructor]	1
Tutorials	1

And the worst aspects of the event for them:

“Worst” Aspect	People reporting this
Lack of social activities / opportunities	10
AWS Polly	1
Slack	1
Not live video	1
Getting lost between the video/tutorial/slides/slack	1

We did not plan social events and this was a mistake on our part. For future Gallantries events in this style we’re planning social activities such that students can perhaps meet each other.

Accessibility

There are a few feedback responses we want to highlight. Some trainees really appreciated being able to revisit earlier materials and do things out of the ‘official’ order, allowing them to deepen their understanding:

To be honest, the asynchronous methodology and the virtuality was extremely convenient to learn. The non-linear learning as I experienced the event, did not only help me to cover details that are normally unseen under deployment and admin but also kindled motivation to master Galaxy at all levels.

The following comment really impacted us. It is one of the reasons we’ve deemed captioning so important: accessibility. Where the instructors talk too quickly or in an accent the listener is not familiar with, this can significantly impact understanding. By moving this event online and giving the students freedom to pause, to rewind, to ask questions whenever they feel like (not at designated periods at the end of a lesson) we enable everyone to have a more enjoyable and better learning experience.

Honestly, this has been far the best training so far. […] The instant support by the trainers has been perfect. This is an important point for me. In my previous in-person trainings sometimes I got lost. Of course I got help, but the class won’t wait (so long). So sometimes I found myself rushing through the following stuff to get back on track again - without learning much. This issue has been totally solved in GAT 2021. I could work in my own pace. Furthermore there has been room for more questions in my opinion in contrast to the in-person trainings.

For trainees who might lose their train of thought by the end of a lecture section and forget their question, now they can just immediately ask it within the chat without “interrupting” the class and have a deep discussion, before moving back to the video and continuing at their own pace. For students who misheard or missed a section, they can review it without interrupting the flow of the class. For students who just wanted to review e.g. the “week overview” diagram to figure out where they are, they can do this at any time without losing out on content.

And a final comment on that topic:

Best - doing this course in my own space with my own tech made this very efficient/comfortable which helped my learning

Often the environment of a small classroom next to unfamiliar people, with a travel laptop can impact the student’s learning experience. If they are used to specific accommodations they have at work or home that help them be more productive and focused, they may be forced to attend without these.

One of the instructors had this view of it:

For students the situation is “I can still hear the instructors but I can pause when I need”, while for us instructors it’s “this is happening but I can’t see anything.” So it’s probably a lot less weird for them than it is for us.

And this likely feeds into the resistance to moving to fully asynchronous events despite their overwhelming benefits in terms of accessibility for all students, regardless of any disadvantages they might experience.

Instructors

Our instructors had positive things to say as well. Simon mentions a known blindspot of students disappearing, we need to find more ways to encourage them to share their successes:

I like some aspects of it. Especially the ability to do a CYOA day. It was less tiring as well. We can definitely cover more people at one and it’s significantly more accessible. However I did miss the social/getting to know the students aspect. The other problem is getting students with problems to speak up and not just disappear quietly.

Saskia is the primary organiser or Smörgåsbord and was taking notes during her role as an instructor in GAT:

I agree with all of the above, it is harder to monitor if anybody is falling behind, but on the other hand it is easier to catch up if you do fall behind than in a synchronous format. The social interaction is the big difference, for Smörgåsbord we are trying to organize some “social sessions” (quizzes, games, chats), we will see if people go for those. The check in calls didn’t really take off, that would have been a good opportunity for some interaction, not sure how we could encourage participation in those in the future.. I definitely like the fact that it is now more accessible for certain regions of the world that would otherwise not be able to participate. And yes, less stress during the week, even easier to rerun (say at GCC), miss the travel and getting to spend a week with all of you of course.

She had an interesting suggestion we’ll look into in future events:

Wonder if we could create “mentoring groups” each instructor gets assigned a smaller group of students in their time zone, they have their own chat channel, can ask them more personally how they’re doing? Maybe it’s easier for some participants to speak up in a smaller group?

And Nicola echoed all of the comments, the issue following along with students is difficult and our progress monitoring system does not capture everything.

Agreed! In general, it has the same benefits and issues of other online training courses. I miss in particular all the social parts with fellow trainers and getting to know new people. Also, as mentioned, it is more difficult to gather the visual feedback of looking at people’ faces to see if they are following or completely lost.

Takeaways

Methodology

1. Captions are mandatory.
2. Asynchronous means accessibility
3. Fully asynchronous works, students love it.
4. Fully asynchronous means anyone can attend regardless of timezone - reduced exclusion based on instructor timezones.
5. The format is less stressful for students and instructors.

Communication

1. Real-time communication was key for students.
2. Slack (or equivalent) works quite well with threaded conversations and separate channels for separate topics.

The Future

With the fantastic feedback and organisation from this event, the Gallantries feels a lot more certain about hosting the largest Galaxy event in history with 700 registrants as of the writing of this post. This methodology works extremely well and scales fantastically.

Thank You

We’d like to say thank you to all of our instructors and sponsors!

Instructors

A huge thank you to the instructors who produced videos (in order video airing order)

• Sergey Golitsynskiy
• Martin Cech
• Helena Rasche
• Catherine Bromhead
• Simon Gladman
• Saskia Hiltemann
• Nicola Soranzo
• Gianmauro Cuccuru
• Anthony Bretaudeau
• Enis Afgan
• Anurag Shankar

As well as the huge number of folks who helped edit and correct captions

• Catherine Bromhead
• Simon Bray
• Simon Gladmann
• Estelle Ancelet
• Beatriz Serrano-Solano
• Anthony Bretaudeau
• Martin Cech

And everyone who helped answer questions in Slack!

Sponsors

This course has received funding from EOSC-Life Second Training Open Call. EOSC-Life has received funding from the European Union’s Horizon 2020 programme under grant agreement number 824087

We are grateful to Jetstream and the Galaxy Team for providing VMs from their allocation TG-CCR160022 for this event.

A significant portion of our infrastructure was graciously provided by Galaxy Australia

Virtual Machines were provided by the German Federal Ministry of Education and Research BMBF grant 031 A538A de.NBI-RBC.