Der letzte Monat im Rückblick Einsichten in die Black Box von Förderungs-Reviews. Bereits kleine Voreingenommenheiten führen zu gravierenden Unterschieden bei Förderquoten. In der hochkompetitiven Umgebung werden so strukturelle Ungleichgewichte verstärkt, vor allem wenn es um institutionalisierte Verzerrungen geht. Nur gucken, nicht anfassen? Im Zuge der Novellierung der Urheberrechts-Richtlinie sieht ein aktuell vorliegender Berichtsentwurf Ausnahmen für […]
In a previous post I’ve already said that I love uWSGI. The main reason? You can do lots of nice tricks in your stack without having to add other layers to it, like for example: graceful reloading.
The documentation from uWSGI is really great, and it covers most of the cases for graceful reloading, however due to our current stack and our auto deployments solution we needed something that integrated well with the so called: Zerg dance.
The Zerg mode is a nice feature from uWSGI that allows you to run your web application passing file descriptors over Unix sockets. As stated on the official docs:
Zerg mode works by making use of the venerable “fd passing over Unix sockets” technique.
Basically, an external process (the zerg server/pool) binds to the various sockets required by your app. Your uWSGI instance, instead of binding by itself, asks the zerg server/pool to pass it the file descriptor. This means multiple unrelated instances can ask for the same file descriptors and work together.
This is really great, as you only need to enable a Zerg server and then you are ready to use it.
As we use Supervisor, configuring uWSGI to run as a Zerg server is really simple:
[uwsgi] master = true zerg-pool = /tmp/zerg_pool_1:/tmp/zerg_master.sock
Then, you configure your web application to use the zerg server:
[uwsgi] zerg = /tmp/zerg_master.sock
And you are done! That will configure your server to run in Zerg mode. However, we can configure it to handle reloading in a more useful way: keeping a binary copy of the previous running instance, pausing it, and deploying the new code on a new Zerg. This is known as Zerg Dance, so let’s dance!
With the Zerg dance we’ll be able to do deployments while the users keep using your web application, as the Zerg server will be always handling those requests properly.
The neat trick from uWSGI is that it will handle those requests pausing them, so the user thinks it’s getting slower, while the new deployment is taking place. As soon as the new deployment is running it moves the “paused request” to the new code and keeps the old copy in case you broke something. Nice, right?
To achieve this situation all you have to do is use 3 different FIFOs in uWSGI. Why? Because uWSGI can have as many master FIFOs as you want allowing you to pause zerg servers and move between them. This feature allows us to keep a binary copy of previously deployed code on the server, that you can pause/resume and use it when something goes wrong.
This is really fast. The only issue is that you’ll need more memory on your server, but I think it’s worthy as you’ll be able to rollback a deployment with just two commands (we’ll see that in a moment).
Configuring the 3 FIFOs
The documentation has a really good example. All you have to do is to add 3 FIFOs to your web application uWSGI config file:
[uwsgi] ; fifo '0' master-fifo = /var/run/new.fifo ; fifo '1' master-fifo = /var/run/running.fifo ; fifo '2' master-fifo = /var/run/sleeping.fifo ; attach to zerg zerg = /var/run/pool1 ; other options ... ; hooks ; destroy the currently sleeping instance if-exists = /var/run/sleeping.fifo hook-accepting1-once = writefifo:/var/run/sleeping.fifo Q endif = ; force the currently running instance to became sleeping (slot 2) and place it in pause mode if-exists = /var/run/running.fifo hook-accepting1-once = writefifo:/var/run/running.fifo 2p endif = ; force this instance to became the running one (slot 1) hook-accepting1-once = writefifo:/var/run/new.fifo 1
After the FIFOs there is a section where we declare some hooks. These hooks will handle automatically which FIFO has to be used in case of a server is started again.
The usual work flow will be the following:
- You start the server.
- There is not sleeping or running fifo, so those conditions fail
- Therefore, once the server is ready to accept requests (thanks to hook-accepting1-once) it moves the server from the new.fifo to running.fifo
Right now you’ve a server running as before. Imagine now you have to change something in the config or you have a new deployment. You do the changes, and start a new server with the same uWSGI config file. This will happen:
- You start the second server.
- There is not sleeping fifo, so this condition fails
- There is a running fifo, so this condition is met. Thus, the previous server is moved to the sleeping fifo and its paused when the new server is ready to accept requests.
- Finally, once the server is ready to accept requests t moves the server from the new.fifo to running.fifo.
At this moment we’ve two servers: one running (the new one with your new code or config changes) and the old one wich is paused consuming only some memory.
Imagine now you realize that you have a bug in your new deployed code. How do you recover from this situation? Simple!
You just pause the new server and unpause the previous one. How do you do it? Like this:
echo 1p > /tmp/running.fifo echo 2p > /tmp/sleeping.fifo
With our auto deployments solution, we needed to find a simple way to integrate this feature with supervisor. In the previous example you do the deployment manually, but we want to have everything automated.
How we have achieved this? Simple! Using two PyBossa servers within Supervisor.
We have the default PyBossa server, and another one named pybossabak in Supervisor.
When a new deployment is done, the auto deployments solution boots the pybossa Backup server just to have a copy of the running state of the server. Then, it gets all the new changes, applies patches, etc. and restarts the default server. This procedure triggers the following:
- Start backup server: this moves the current running PyBossa server to the pause fifo, so we’ve a copy of it.
- The backup server accepts the requests, so users don’t see anything wrong.
- Autodeployments applies changes to the source code, updates libraries, etc.
- Then, it restarts the default PyBossa server (note: for supervisor the paused PyBossa server is running).
- This restart moves the previous backup server to the pause fifo (it has the old code running), and boots the new code into production.
If something goes wrong with the new changes, all we have to do is pause the current server and resume the previous one.
This is done by hand, as we want to have control over this specific issue, but overall we are always covered when doing deployments automatically. We only have to click in the Merge Button of Github to do a deployment and we know a backup binary copy is hold on memory in case that we commit an error.
Moreover, the whole process of having uWSGI moving the requests of users from one server to another is great!
We’ve seen some users getting a 502, but that’s because they ask for a request when the file descriptor is being moved to the new server. Obviously, this is not 100% bullet proof, but much better than showing to all your users a maintenance page while you do the upgrade.
We’ve been using this new work flow for a few weeks now, and all our production deployments are done automatically. Since we adopted this approach we’ve not have any issues, and we are more focused only on developing more code. We employ less time handling deployments, which is great!
In summary: if you are using uWSGI, use the Zerg Dance, and enjoy the dance!
This is old, unpublished news/post that I never got around to posting for some reason… In the last week of March, I started to think about how the Open Science Framework (OSF) can foster a non-technical community. At first, I thought about only of advocacy and teaching of the scientific process. But after the response […]
Center of Open Science
How do you translate properly your product into different languages? More importantly, how do you do it involving your own community?
The answer is easy: using a crowdsourcing solution like PyBossa.
Translating PyBossa using PyBossa
Since the creation of PyBossa, I’ve translated it to Spanish. Other languages, like French, were added by a volunteer. However, these translations usually get outdated as PyBossa was updated with new strings. These solo efforts, usually end up in a translation that’s not updated, and you end up with a mix of translated strings.
For these reasons we decided to eat our own dog food, and I created a crowdsourcing project to translate PyBossa using PyBossa. Why? Because PyBossa uses the open standard Gettext for its translations, and each string could become a task in a PyBossa project.
Also I loved the idea that anyone, even without an account, can help in the translation. The current platforms usually need an account to just translate a few strings, and that’s usually too much for users who want to see the product they use in their own language. Obviously some people will add fake translations, but that’s not an issue as the crowd will help to clean the bad ones and keep the best one.
As I started working on it, I realized this could be very useful not only for me and PyBossa but also to anyone using the Gettext technology in their projects. Thus, I created a PyBossa template project that anyone can re-use and adapt today to translate their own projects.
The Translation Template Project
The template can be used in any PyBossa server, so if you don’t have one, don’t hesitate and go to Crowdcrafting to create an account and start using it.
The translation template is very simple. It has been designed to have two phases:
- The Translation: 3 people translate the same string.
- The Voting: 5 people vote for the best translation of the 3 translations.
The most voted, it’s the one that it’s going to be used as the final translated one.
As you can see the community of your project would be involved in translating but also in selecting the best translation for them. This will ensure that your audience will have a better understanding about the text you write, leading to better results in engagement.
1. The Translation phase
The first thing you need to do is to download the template. Then, install the required tools (see the README file for more information), and you will be ready to start translating your project.
Then, all you have to do is get your PO file (it’s a text file with the string to get translated from for example English to Spanish). Once you have it, you will pass it to PBS -our PyBossa command line tool- that will convert untranslated strings to tasks for your PyBossa project:
pbs add_tasks --task-file=messages.pot --tasks-type=po --redundancy=3
This will add the untranslated strings as tasks to your PyBossa project. Each string will be shown to 3 different people, so you get 3 translation for your own project. You can increase or reduce it as much as you want. It’s up to you to decide.
When all the strings have been translated, you can move to the next phase if you want: the voting phase.
2. The Voting phase
In this phase, the 3 previous translations will be shown to people and they’ll select the best one for them. The most voted one will be the final translation for that string.
How do you move from one phase to the next one? As simple as this. First we create the voting project:
pbs --project project_voting.json create_project pbs --project project_voting.json update_project
Secondly, we get the translated strings and pass them to the new voting project:
python vote.py pbs --project project_voting.json add_tasks --task-file=/tmp/translations_voting_tasks.json --redundancy=5
Then, 5 people will vote on which is the best translation. When all the strings have been curated by your community, in other words when the project is completed, all you have to do to create the final translation file is running the following command:
Copy the new created file into your translations project, and you’ll be done! As simple as that.
Yes, PyBossa also supports Firefox extensions. Thus, if you are writing a Firefox extension and you want to translate it to different languages, you can use PyBossa too. It’s pretty similar and you have all the documentation about it here.
With our PyBossa translation template anyone can translate their open source project with their community, involving them not only in the translation but also curating which is the best translation for every string.
Thus, don’t get lost in translation anymore!
Because I started my Online volunteering in Ubuntu Community, that means that is part of the Open Source community and there is something that I had a feeling that I worked towards (along with other things, of course). That something is the concept of membership. It’s the benefits of being recognized for significant, sustained, continued, […]