Alex Moreno


A Drupal Developer in London

Ansistrano, deploying Drupal with Ansible

Last months have been pretty busy here at BBC Worldwide with some of the projects we've been involved in.

In my case, one of the main areas I’ve been trying to improve (apart of the daily busy routine) is our devops / CI approach. The previous approach was using Chef, but it was not really adopted by the developers (which meant that after some months, the system was broken) and we were lacking some fundamental things (like integration with the production system, a real CI flow, etc…).

On the other hand, my previous experience with puppet was that the system grew too much to be easily maintained, specially if you don’t have a big dedicated devops team. To sum up, during last months research I was pretty amazed with Ansible, it's amazingly simple, very flexible and a truly joy to maintain. So, to me, the way forward was Ansible.

So, we started rewriting some of the old broken features in Chef and, sooner than later, we found ourselves reinventing the wheel. Some features were custom things, like triggering "drush cc all”, executing grunt, phpcs, etc... but obviously lots others were not… well, we are not the first team in needing an apache and mysql database, are we?

And deployment is one of these things that I was not too keen to reinvent. It would be pretty simple with Ansible, to be honest, but why not use something that is already tested, works well and at the same time give some love back to the community? Enter Ansistrano.

Ansistrano is a deployment tool based in Ansible. The relation with Capistrano ends just here, with the name similarity (no code base was harmed during the process). For more information about Ansistrano please visit the official repository: https://github.com/ansistrano/deploy

- Step 0. Install ansistrano (deploy and you’ll probably be interested in the rollback too):

ansible-galaxy install carlosbuenosvinos.ansistrano-deploy carlosbuenosvinos.ansistrano-rollback

- Step 0. Clone the deploy repo in the folder you’ll be using the app. Let’s say we’ll be working in ~/my-app-build :

git clone https://github.com/ansistrano/deploy.git ansistrano-deploy

Notice that we are cloning the deploy repository.

We’ll be using the latest stable release, list tags:

vagrant@vagrant-ubuntu-trusty-64:~/my-app-build$ git tag
1.0.0
1.1.0
1.2.0
1.3.0
1.4.0
1.4.1

at the moment, the latest stable release is 1.4.1, so:

git checkout 1.4.1

Step 1. Now, the ‘tricky part’. We are going to write a file where we’ll execute all the magic. Let’s call it build-my-app.yml:

---
- name: Deploy example app to local.myapp.com
  hosts: all
  vars:
    ansistrano_deploy_to: "/var/www/myapp.local"
    ansistrano_keep_releases: 3
    ansistrano_deploy_via: "git"
    ## GIT pull strategy
    ansistrano_git_repo: git@bitbucket.org:myapp-in-git.git
    ansistrano_git_branch: develop
    ansistrano_git_identity_key_path: ""

  roles:
    - { role: ansistrano-deploy }

Notice that the role we have at the end is just the deploy repository cloned from Ansistrano (https://github.com/ansistrano/deploy). You can change the name, as I've done in this example (ansistrano-deploy) but you'll have to upgrade the roles variable in your yml (again, as shown in this example).  

Step 2. Run deploy:

ansible-playbook -i "localhost," -c local build-my-app.yml -vvv

Instead of specifying localhost you could also use a hosts file, as recommended in the official documentation. This is also going to give you the freedom of deploying your app not just in your local environment, but potentially in any remote server, i.e. aws, digital ocean, etc...

Step 3 (optional). From here, the limit is the sky. In our case, we have some complex deployments running grunt, drush, clearing cache… The advantage here is that ansible is pretty flexible, so you could for example add your own tasks in your deploy file (build-my-app.yml), i.e.:

tasks:
  - include: goodfood/tasks/apt.yml

The right, official way of doing this is using the hooks that ansistrano itself offers you:

  # Hooks: custom tasks if you need them
  ansistrano_before_setup_tasks_file: "{{ playbook_dir }}/<your-deployment-config>/my-before-setup-tasks.yml"
  ansistrano_after_setup_tasks_file: "{{ playbook_dir }}/<your-deployment-config>/my-after-setup-tasks.yml"
  ansistrano_before_update_code_tasks_file: "{{ playbook_dir }}/<your-deployment-config>/my-before-update-code-tasks.yml"
  ansistrano_after_update_code_tasks_file: "{{ playbook_dir }}/<your-deployment-config>/my-after-update-code-tasks.yml"
  ansistrano_before_symlink_shared_tasks_file: "{{ playbook_dir }}/<your-deployment-config>/my-before-symlink-shared-tasks.yml"
  ansistrano_after_symlink_shared_tasks_file: "{{ playbook_dir }}/<your-deployment-config>/my-after-symlink-shared-tasks.yml"
  ansistrano_before_symlink_tasks_file: "{{ playbook_dir }}/<your-deployment-config>/my-before-symlink-tasks.yml"
  ansistrano_after_symlink_tasks_file: "{{ playbook_dir }}/<your-deployment-config>/my-after-symlink-tasks.yml"
  ansistrano_before_cleanup_tasks_file: "{{ playbook_dir }}/<your-deployment-config>/my-before-cleanup-tasks.yml"
  ansistrano_after_cleanup_tasks_file: "{{ playbook_dir }}/<your-deployment-config>/my-after-cleanup-tasks.yml"

where playbook_dir is the folder where your yml file lives.

That's it. Simple, elegant, and really powerful, I love it :-). Next episode, we'll talk about the rollback artifact.

Notes: Amended Step 0, as you don't need to install the playbook through the ansible galaxy, just cloning would do. Thank you Carlos Buenosvinos.

Software Craftsmanship

Software Craftsmanship is a mindset— a lifestyle that many professional developers adopt. Software craftsmen live and breathe software. They see software as a craft and are committed to do whatever they can to master their craft

Ingeniería y cafe

"Un ingeniero es una máquina que convierte café en diseños, construcciones y líneas de código." Parafraseando a Alfréd Rényi

categorias: 

Bad code

Don’t comment bad code— rewrite it.

Brian W. Kernighan and P. J. Plaugher1

Development phrase of the day

If we do things that way, we are going from Spagetti Code to Lasagna Code.

Howto use Lazy Loading in Symfony2 (and Drupal 7 / 8)

Lazy Loading is a software design pattern which basically allows our classes and objects to be a lot lighter. Faster code, faster applications, less memory used in runtime. The great Martin Fowler says about Lazy Loading:

An object that doesn't contain all of the data you need but knows how to get it.

http://martinfowler.com/eaaCatalog/lazyLoad.html

The key is that your "real" objects are not going to be instantiated unless they are going to be used. How that can be achieved? Easy, with what it is called a Virtual Proxy. As in every project, you can always code your own implementation, but the key about design patterns and software engineering is to reuse as much as possible. And regarding Lazy Loading, Symfony 2 has a nice implementation called Proxy Manager Bridge, which does exactly that in Symfony 2.

Recently we discovered that our implementation of Lazy Loading was not working properly in Drupal 7. We were using a custom module which we created in Capgemini, Drupal Symfony Inject, to inject our own implementations of Symfony 2 classes in Drupal. The module works great, but for some reason the Lazy Loading was not working, the objects were always having the real implementation of the object itself instead of the proxy (for more info, check the documentation in Symfony 2 about Lazy Services).

The solution is relatively easy (not as much as finding it), and I commited a patch for that, so Drupal Symfony Inject should not have that problem anymore: https://drupal.org/node/2287081.

A different question is, how to use Lazy Loading in our projects when we are not using Symfony 2 framework but its components. One of the errors which we were doing in the Drupal Symfony Inject module was that we were not loading the Proxy Manager Bridge, which makes the integration of Proxy Manager, your code and some other Symfony 2 components.

At that point we were quite lost, not many people has done before that integration between Drupal and Symfony 2 Lazy Loading, so we had very few people to ask, even inside Capgemini itself, and even fewer documentation to consult. Fortunatelly there are some interesting pieces of code in every well designed software. I am talking about test units: https://github.com/symfony/ProxyManagerBridge/blob/master/Tests/LazyProx...

The most interesting part in lines 34 and 37:

$builder->setProxyInstantiator(new RuntimeInstantiator());

That part is essential, it will instantiate the Proxy Manager and substitute your Object with an instance of that Proxy.

We just need to do one last step, indicate that our object will use Lazy Loading:

$builder->getDefinition('your_class')->setLazy(true);

From this point, the next time you load your object:

$builder->get('your_class')

if you ask for the class of this object it will identify itself as a Lazy Proxy:

\ProxyManager\Proxy\LazyLoadingInterface

 That state will remain until you ask the object to return something, basically when you use one of its methods:

$your_object->doSomething();

Now that object will say that he is the real class, something like that:

\ProxyManagerBridgeFooClass

 

More information:

Profiling PHP and Drupal with XDebug (In OSX)

Problems with a rogue app you built (maybe not you)?

Just some links which I found useful in order to debug/profiling your app:

Run it in you Mac with OSX:

Sometimes you need your profiler to be executed everytime. You can do that with this:

xdebug.profiler_enable=On

This is my whole xdebug.ini:

; Enable xdebug extension module

zend_extension=/usr/lib64/php/modules/xdebug.so

xdebug.remote_enable=1

xdebug.remote_host=192.168.33.1

xdebug.remote_port=9000

xdebug.remote_mode=req

xdebug.remote_connect_back=0

xdebug.profiler_enable=On

xdebug.profiler_output_dir=/www/profiling_data

xdebug.profiler_enable_trigger=1

 

Other general links:

 

Composer and Symfony2 in Drupal 7

categorias: 

Developer definition

Programmer: an organism that turns caffeine and pizza into software :-D

categorias: 

Using Symfony Dependency Injection in Drupal 7: Software Engineering in Drupal

I still remember in a previous job discussing with a partner about doing things the proper way, using coding standards, commenting what you are doing, ... giving your code some love in definitive.

I'm still amazed of his answer, basically he was refusing to doing things the right way (the only way) because using his way he was a lot quicker... Well, probably, but on the meantime modifying some of the crap we had there (not just him, I have to say) could take us like 4 to 10 times more time.

That's basically because you needed to understand what was going on that code... and I promise you, when you have a single function of around 3000 lines, no comments, things happening without too much logic, no standards, dozens of lines commented... that is not easy at all, even for the owners of that code. That's why you use Coding Standards, code reviews, Pull Requests and all that stuff in serious environments of Software Engineering.

Anyway, developing using the Object Oriented paradigm is quite a lot more organized than doing it using simply .modules and .inc files in Drupal. It does not matter how organized you are, at the end you'll end with lots of files and functions in a nightmare to understand for third developers. And, when you are working in Enterprise environments, with lots of people sharing and creating code, things like coding standards and doing things the proper way becomes a very serious thing.

Solution? As I said, using Object Oriented programming paradigm. In Drupal is easy enough since Drupal 7 allows you to add directly classes included in .php, .inc/* files. Simply do:

files[] = lib/MyClass.php

Well, that's easy enough... but there is a better more sophisticated way of doint it: Symfony Dependency Injection.

There are some good advantages of doing this that way. First and maybe most important, you are starting to be ready for Drupal 8. Second one, you objects can use Lazy loading. That's a great thing, basically means that your objects will be using the Proxy Pattern, which allows your code to load things only when they are going to be needed.

That's a good trick to improve your Drupal site performance from your own code (a big big percentage of performance problems are derivated from bad coding practices).

Want more advantages? Did I say that you make your code ready to Drupal 8? Well, that's just the begining. Starting to use Symfony components like Drupal Symfony inject open us a world of thousands of bundles ready to use from Symfony, and just using them through Composer. Win win :-).

We will need basically two modules, Drupal Symfony inject which a college is developing here, in Capgemini, and Composer Manager as it is a dependancy:

It is important you install them using drush, so the vendors folder will be created for you (with all the dependencies that the module has):

drush pm-enable drupal_symfony_inject --y

Once installed, DSI makes you available three new hooks:

  • _namespace_register()
  • _symfony_yaml_config()
  • _symfony_yaml_config_params()

The first one, _namespace_register, allows you to register your new classes like that:

Now we will need to add the config.yaml, where we will declare the classes we will be using, parameters, etc...

Now you need to define your classes in the yaml file. Create the file in config/yaml/config.yaml and declare there the classes you'll be using:

And finally, the parameters:

That's it, Drupal is ready to use the Symfony Dependency Injection component. Lets move to create a new class, for example, CruiseHunter:

That's one of the Abstract classes I use in CruiseHunter from which I inherit to fetch the different contents the Cruise crawler uses. It is a lot more organized, it declares a common mechanism to fetch information from the crawler, etc...

Well, now that we have the class, we need to be able to import it and use it. And here is where the Dependency Injection appears:

Once you have all in place, you have to rebuild the symfony components you have just built:

drush rsc

If you see some error, you made a mistake, probably in your yaml. If you see something like this:

Symfony DI container compiled and cached successfully !!!

Congratulations, you are ready to use your new Symfony components :-).

Related articles: