I as the LA Technical coordinator

Mainly:

• I try to give support in our slack channel to all questions
• I introduce new Living Atlases (LA) developers to our community and best practices
• I take care of our documentation

and

• I develop and maintain some utilities for the community and others things like LA debian packages
• I take care of the ALA internationalization process
• I try to contribute to the ALA software as much as I can

I also take care of gbif.es infrastructure

• that helps me also, to do my job as technical coordinator
  • learning in deep the ALA software
  • testing things
  • solving real usage problems
  • but not always…

What I try

• To follow ALA as much as possible (deployment methods, configurations, software usage)
• Try to return back to ALA repositories also as much as possible (feedback, issues, code, translations, documentation, thanks!, whatever)

My code contributions to the ALA repositories

• gbif.es probably has only ≈5 lines of customized code (easy to maintain)
• Instead, I did ≈350 Pull Requests to ALA since I joined this community 3 years ago,
• many of them are not big functionalities,
• mainly they resolve issues useful for the LA community.
• In other words, less problems that others have to solve in their LA Portals

What I found solving these issues

• That sometimes some issues were solved by other LA portals previously,
• but didn’t send back their fixes to ALA codebase
• For instance: Other portals installed previously some services than us (alerts, sds, …) but didn’t return back how they did it or how they patched ALA

Quoting Eric S. Raymond

No problem should ever have to be solved twice

Eric who? ESR in 1 min:

… the problem about forking, IMHO

• ALA is a huge 12 years software project …
• … that it's maintained by a huge technical team (≈1X members)
• Custom code changes are sometimes quite easy to do, but quite hard to maintain
• Other non Australian LA technical teams are quite small
• Many like gbif.es, has only one IT person (well I’m ½), or even they don't have an IT permanent team (like Canadensys or Andorra)
• These small IT teams cannot maintain well a customized version of the ALA code

Forks are hard to maintain...even for Google

they forked the Linux kernel for Android, but:

Quoting again Eric S. Raymond

The most important characteristic of a fork is that it spawns competing projects that cannot later exchange code, splitting the potential developer community. Forking [in Free/Libre/Open Source Software Projects] is considered a Bad Thing

Maintenance problems arose later

• Some LA portals has serious issues updating their portals because of their old customizations,
• their customizations are difficult to rebase/merge, to test (again), even to justify the effort,
• they are trying to come back to the ALA codebase

My personal recommendations

• follow the (ALA Team) pack
• Share alike, return back,
• I like the cyclist pack metaphor and the wind: when you are cycling alone you have to fight alone with the wind
• A small team should not itself customize a lot ALA (this is the easy part like: “I’ll change Australia by my country name here and there in my local computer”)
• because later the hard part is maintain these customizations trying to follow ALA software changes over time

A google docs anecdote

• Some years ago we were preparing a request for fundings using google docs
• Two days before the deadline some colleague send us via email a document with her modifications (she was quiet during all the writing process)
• [Horror song] we couldn't identify and merge her modifications
• The same happens in our community sometimes using git:
• someone forks a ALA module, and some months (or years) later, they say, "here I am with my version!"

... instead of just forking I recommend

• Fork, but try to generalize your software customizations and send Pull Requests to ALA repositories ASAP
• Try to make things configurable via variables, instead of hardcoding local requirements
• so your code can be useful to other LA nodes
• and also the code is maintained by all of us,
• and we use and try to improve the same code every time,...

that is,

• Let’s share our improvements with the rest of the community,
• Let's fight the wind in rotating turns

Preparing the ground for others

If I solve an issue in gbif.es or I achieve to install or update some service:

• I think in the rest of the LA community
• and I try to imagine what I can do to make others do the same job without much effort
• For instance, I spent a pair of months trying to figure out how to install CAS5 in gbif.es,
• but now it’s something that anyone can deploy in 5 minutes

How do I prepare the ground?

• with documentation
• or code (something I prefer):
  • patches to ALA software
  • devops code
  • LA tools to help with our daily tasks

Some concepts: ala-install

• The official ALA devops repository to automatic deploy and maintain LA portals
• In other words: this code installs automatically the ALA software
• ≈ 10,000 admin tasks to deploy a small LA portal
• You can deploy and maintain your LA portal with the same deployment code that ALA and other LA portals use
• It uses ansible under the hood
• I test ala-install to detect issues (via a jenkins job that installed ~600 LA testing portals so far)

Some LA Tools I develop...

LA Ansible Generator

• if ala-install are the cooking recipes, ansible your cooking robot, the inventories are your local ingredients
• Before: we have to write our ansible inventories from scratch or ask someone for a sample and adapt it (quite effort)
• Now: with a wizard, we can autogenerate all these inventories and some helper to run ansible. But also maintain these inventories with new versions of the generator and ala-install. So your LA configuration keeps updated with new software releases.
• It's a command line tool but I did also a web version
• More

LA Data Generator

• It gets the ala-install + some inventories and generates the typical /data configurations of a LA Portal
• useful for development but also for docker based LA portals
• More

ala-i18n packaging

• Before: we had to send Pull Requests with our translations to ALA, get merged, be released, install the new release with our translations
• Now: we'll do the translations in crowdin, and automatically a new ala-i18n debian package is built and released, so we can install this new package in our servers.
• The new i18n messages are also automatically uploaded to crowdin, so translators can do their job on each new release
• I’ve just added support for biocollect/ecodata
• More info

LA Base Branding

Brandings for new LA deployments. Goals:

• Well integration with ALA modules trying to avoid javascript libs conflicts and similar
• Well integration with CAS Authentication
• Multilingual
• Modern javascript code without lost compatibility with old browsers (npm install somedep!)
• Many sample themes
• SBDI uses it as a base for its branding

LA Base themes

Some LA debian packages

• apt install ala-i18n # commented above
• apt install la-pipelines
• apt install ala-namematching-service
• apt install ala-sensitive-data-service
• apt install ipt # wait this is not ALA
• So ansible ala-install based portals and docker based ones (as SBDI) can use the same [debian] install code.

But this was not enough

• The deployment is still difficult for newcomers
• It requires some GNU/Linux deployment environment with all the tools (like ansible) well configured
• There are many options
• It requires some sysadmin knowledge
• It's not so clear how to maintain or upgrade a LA portal

LA Toolkit

About GBIF.ES deployment & maintenance

• Deployed using ala-install (without local changes)
• with inventories created and maintained with the living-atlas ansible generator and now with the la-toolkit
• As a result: our ALA software versions are quite updated following ALA releases
• We are open to use docker or other deployment solutions in the future, but only when docker is supported by ALA

Some recent sample

• ALA team install and maintain biocollect/ecodata with ala-install as the rest of their infraestructure
• Recently I gave support to biocollect/ecodata in the LA Toolkit and related (although we don’t use biocollect in gbif.es nowadays)
• The past week, some newcomer started to install a testing LA Portal
• 3 days later he had biocollect/ecodata and the rest of ALA modules deployed successfully

Thank you indeed