Saturday, May 21, 2016

Some guidelines for writing better and safer code

Recently, I came across some code of a web application that, on brief inspection, was vulnerable to XSS and SQL injection attacks : the SQL queries and the HTML output were not properly escaped, the input variables were not sanitized. After a bit more reviewing I made a list of measures and notified the developer who quickly fixed the issues.

I was a bit surprised to come across code that was very insecure, which took the author only a few hours to drastically improve with a few simple changes. I started wondering why the code wasn't of better quality in the first place? Did the developer not know about vulnerabilities like SQL injection and how to prevent them? Was it time pressure that kept him from writing safer code?

Anyway, there are a few guidelines to write better and safer code.

Educate yourself

As a developer you should familiarize yourself with possible vulnerabilities and how to avoid them. There are plenty of books and online tutorials covering this. A good starting point is the Top 25 Most Dangerous Software Errors list. Reading security related blogs and going to conferences (or watch talks online) is useful as well.

Use frameworks and libraries

About every language has a framework for web applications (Drupal, Symfony (PHP), Spring (Java), Django (Python), ...) that has tools and libraries for creating forms, sanitizing input variables, properly escaping HTML output, handling cookies, check authorization and do user and privileges management, database-object abstraction (so you don't have to write your own SQL queries) and much more.
Those frameworks and libraries are used by a lot of applications and developers, so they are tested much more than code you write yourself, so bugs are found more quickly.

It is also important to regularly update the libraries and frameworks you use, to have the latest bugs and vulnerabilities fixed.

Code review

More people see more than one. Have your code reviewed by a coworker and use automated tools to check your code for vulnerabilities. Most IDEs have code checking tools, or you can implement them in a Continuous Integration (CI) environment like Jenkins, Travis CI, Circle CI, ... to check your code during every build.
A lot of online code checking tools exist that can check your code every time you push your code to your version control system.
There is no silver bullet here, but a combination manual code review and automated checks will help to spot vulnerabilities sooner.

Test your code

Code reviewing tools can't spot every bug, so testing your code is important as well. You will need automated unit tests, integration tests, ... so you can test your code during every build in you CI environment.
Writing good tests is an art and takes time, but more tests means less possible bugs remaining in your code.

Coding style

While not directly a measure against vulnerabilities, using a coding style that is common for the programming language you are using, makes your code more readable both for you, the reviewer and future maintainers of your code. Better readability makes it easier to spot bugs, maintain code and avoid new bugs.


I guess there are many more ways to improve code quality and reduce vulnerabilities. Feel free to leave a comment with your ideas.


Friday, January 01, 2016

This was 2015 and plenty to do in 2016!

2015 was an amazing year, learning a lot and making some progress in my Open Source development and climbing activities.
Buildtime Trend keeps growing, with Angular and a Facebook Open Sourced project as new users, improving my Python and JavaScript skills, setting up a CherryPy based service on Heroku, backed by a Celery/RabbitMQ task queue to make the service more responsive.

I have no real resolutions for 2016, I'll just spend my time on climbing and Open Source software development, learning new skills along the way and putting them into practice :

  • use Ansible (or another configuration management tool) to provision a Vagrant based development environment for Buildtime Trend
  • start using Django to add user management to Buildtime Trend as a Service
  • learn how to climb safely in less equiped areas using friends, nuts, and other mobile protection
  • apply the lessons learned form "The Rock Warrior's Way' while climbing
  • visit a few conferences : FOSDEM, 33C3 and Newline, and maybe some more : LinuxTag, DebConf, FossAsia, KeenCon, ...
  • do some improvements to my house

Plenty to do in 2016! I wish anyone an joyful year full of insights and opportunities to learn and improve.
And remember, it's all about enjoying the journey!


Here are some highlights from 2015 :
  • Celebrated New Year in Lisbon.
  • Reached a 365 day commit streak contributing to Open Source projects, with 2300+ commits over that period.
  • visited FOSDEM 2015, another great weekend of Open Source enthousiast meeting and sharing knowledge in Brussels, with over 500 speakers and 5-10.000 visitors. Happy to meet Eduard and Ecaterina again who came over from Romania, and many others.
  • Buildtime Trend was mentioned in the Black Duck newsletter 
  • Buildtime Trend made it to the top 3 of hot projects on OpenHub
  • Reached the most active developers top 10 on OpenHub
  • Released Buildtime Trend v0.2 and launched a Heroku app hosting the service.
  • Visited Cork and Dublin with Sofie, attending Jeroen's PhD graduation ceremony and meeting Rouslan and his friends.
  • Attended Newline 0x05 and did two talks : Buildtime Trend : Visualise what's trending in your build process and What I learned from 365 days of contributing to Open Source projects
  • Ended my commit streak after 452 days
  • Went on a climbing trip to Gorges du Tarn
  • Flashed my first 6a lead climbing on rock.
  • Traveled to the US, East coast this time, visiting Washington DC, meeting Lieven H and Wim, exploring New York City with Tine, Lukas and Marie-Hélène.
  • One-day climbing trip to Freyr with Peter.
  • And another climbing trip to Beez with Lieven V, Ben, Patrick and others.
  • 4th blood donation, convinced Tine to join me for her first donation! Well done!
  • Deployed a Celery/RabbitMQ based worker to Buildtime Trend as a Service on Heroku, taking some load off the webservice and improving the response times.
  • Climbing trip to La Bérarde, with Bleau, doing my first multipitches (Li Maye laya and Pin Thotal) with Mariska and lead climbing a few 6a+ routes. Weather was great, atmosphere was superb, climbing was fun!
  • Went to Fontainebleau for the birtdayparty of Andreas. Great fun, nice people, lots of routes. Finished my first red route in Fontainebleau.
  • Travelled to California and made roadtrip from San-Francisco, to Yosemite, over Tioga Pass to Mojave Dessert, Red Rock Canyon, Las Vegas, Zion National Park, Bryce Canyon, Grand Canyon and flying back from Phoenix to San-Francisco. I did some climbing and hiking, took a climbing course on using cams and nuts. On the way I met a lot of nice people, with whom I had interesting conversations.
  • Released Buildtime Trend v0.3
  • Finished online Stanford University course Algorithms: Design and Analysis, Part 1
  • Read The Rock Warrior's Way, a must read for any climber
  • Visited 32C3 in Hamburg, 4 days of lectures, writing software and talking to other developers. It was amazing, next year again!

Tuesday, November 24, 2015

Buildtime Trend v0.3 is out

Visualise what's trending in your build process

Buildtime Trend Logo
I'm happy to inform you that Buildtime Trend v0.3 is released. Those of you using Buildtime Trend as a Service had a running preview of all the new features :

  • introduction of a worker queue to make processing build job logs more scalable
  • dashboard chart data can be filtered on build properties 
  • several new dashboard charts and layout improvements
  • enable Keen.io query caching to improve chart loading speed
  • the dashboard takes url parameters to set the refresh rate and the default settings for time interval and filter properties
  • a statistics dashboard is added to monitor usage of Buildtime Trend as a Service
Dashboard example
Dashboard example
More new features, improvements and changes can be found in the release notes and the Changelog files of the project components :
You can check out the dashboards of the projects that are already using Buildtime Trend as as Service :


Do you want to enable Buildtime Trend for your the build process of your project on Travis CI? It is easy to set up.

Buildtime Trend as a Service is currently available for free for Open Source projects, thanks to the kind people of Keen.io.

Donate

If you like Buildtime Trend, you are welcome to support the project, by making a donation. Donations will help pay for the hosting and support further development.

You can help make the project better : we welcome any kind of contributions.

Tuesday, April 21, 2015

Gorges du Tarn 2015

It was an amazing week, climbing in Gorges du Tarn with Bleau Climbing team during the second week of the Easter holiday.
Beautiful weather, nice people, good atmosphere, a lot of climbing, some personal bests and climbing improvements on both a physical and mental level.

Some impressions :

  • Flashed my first 6a lead climbing on rock : Nique le lière in the Figues au Cul sector.
  • Cooked spaghetti for approx. 50 people with a great team (Peter, Lieven, Hanne, Anneke, Jasper and Tim)
  • Finished a lot of 5s on sight or flashed them.
  • Discovered that my tent isn't waterproof anymore. Luckily there was a bridge that kept part of the camping site dry, so I moved my tent there during the rainy night. Yes, I slept under a bridge. ;)
  • Almost finished a 6a+ on Noir Désir and started working in a very exciting 6c on La Muse, two projects for next time, so I know what I'll be training for the next few months. :)
Great trip, looking forward to the next one!

Sunday, February 22, 2015

Buildtime Trend v0.2 released!

Visualise what's trending in your build process

Buildtime Trend Logo
What started as a few scripts to gain some insight in the duration of stages in a build process, has evolved into project Buildtime Trend, that generates and gathers timing data of build processes. The aggregated data is used to create charts to visualise trends of a build process.

The major new futures are the support for parsing Travis CI build log files to retrieve timing data and the introduction of the project as a service that gathers Travis CI generated timing data, hosts a dashboard with different charts and offers shield badges with different metrics.

Try it out!

The hosted service supports Open Source projects (public on GitHub) running their builds on Travis CI. Thanks to the kind people of Keen.io hosting the aggregated data, the hosted service is currently available for free for Open Source projects.
Get started! It's easy to set up in a few steps.

A bit more about Buildtime Trend

Dashboard example
Dashboard example
Buildtime Trend is an Open Source project that generates and gathers timing data of build processes. The aggregated data is used to create charts to visualise trends of the build process.
These trends can help you gain insight in your build process : which stages take most time? Which stages are stable or have a fluctuating duration? Is there a decrease or increase in average build duration over time?
With these insights you can improve the stability of your build process and make it more efficient.

The generation of timing data is done with either a client or using Buildtime Trend as a Service.
The Python based client generates custom timing tags for any shell based build process and can easily be integrated. A script processes the generated timing tags when the build is finished, and stores the results.
Buildtime Trend as a Service gets timing and build related data by parsing the logfiles of a buildprocess. Currently, Travis CI is supported. Simply trigger the service at the end of a Travis CI build and the parsing, aggregating and storing of the data is done automatically.

The aggregated build data is used to generate a dashboard with charts powered by the Keen.io API and data store.

Check out the website for more information about the project, follow us on Twitter, or subscribe to the community mailing list.

Tuesday, January 13, 2015

What I learned from 365 days of contributing to Open Source projects

Today I reached a 365 day commit streak on GitHub, with over 2300 commits to Open Source projects in that period. In this post I'd like to share my experiences during this past year and the lessons I learned.

Github contribution overview

It started one year ago, on January 14th, 2014, the day I returned from a two week trip to Malaysia. I didn't take a laptop or smartphone on that trip, in order to be 'disconnected' from PC, internet and E-mail for some time. I've taken a habit to have a 'unplugged' holiday about once a year.
I had a streak of over 100 days running before I left on holiday, and had reached 2000+ commits during that year, so I was eager to start committing again, to keep the running total of commits close to 2000 and to start building a commit streak again.
I don't remember if I had a clear goal at that time of the total consecutive days of committing to Open Source projects I aspired to reach. I guess at first I wanted to match the previous record and see how much further I could get.

At the end of June, I got inspired by Josh (@dzello) from Keen.io who had pledged to commit to Open Source projects for 365 days in a row. I had an extensive streak going on at that time, so I decided to try and reach a 365 day streak as well.

Until then it had been fairly easy to keep the streak going. Doing at least one commit every day is not that hard, and I usually did more than one. I was working on my Android app and I started working on what I called 'my side project' at first. In the last few months my focus has shifted to that 'side project', making it my main project basically, but that's a different story.
So I had plenty to do and I was well motivated to work on my projects regularly, so it was easy to keep committing daily.

Until summer I didn't do any long trips, so I was either at home at some point during the day or had my laptop with me (when going to Berlin for LinuxTag, for example) so finding a few minutes every day to do at least one commit wasn't a big challenge. Although sometimes, it required some planning.
If I had a social, cultural or sports activity planned in the evening after work, I planned for an 'easy' commit on those days, usually cleaning up some code, fixing coding style, writing a small test, improving documentation or doing a few translations. I kept the bigger work, implementing a new feature, figuring out an API or framework, or doing some bigger refactoring for days when I had more time available.

Then summer arrived and I planned to go on a climbing trip for a week. I was in doubt if I would have time and opportunity to keep committing during the trip. But in the end I decided to take my laptop and give it a try. I worked on bash script to crop and scale Android screenshots that week, something I could easily develop and test without access to internet. On some days I barely managed to contribute, finishing the commit only a few minutes before the end of the day, but in the end I managed to keep the streak going.

With this hurdle taken I imagined reaching the 365 day goal was achievable. I didn't know back then I was to go on a few long weekends to Fontainebleau during the Fall, but again I took my laptop with me on the trip and found time to contribute some code.

In the end of October I went to California for the Google Summer of Code 10 year Reunion and I had the opportunity to meet Josh, and Justin, also from Keen.io. Josh  published a blogpost by that time explaining why he ended his commit streak at Burning Man, and why he wasn't planning on starting the streak again.

I read his post, and it got me thinking. Is this continuous streak a good thing? Sure, it was a motivation on its own, you want to keep going because you don't want to break the streak, you'd have to start all over again for a long time to reach the same number of consecutive days.
Some days you don't have time, or you've had a rough day and don't feel like turning on your PC and doing the required commit for the day, but would rather do something else, forgetting about that ongoing streak.

But I decided to go for it and finish the pledge to reach 365 days of committing to Open Source projects. I found out that my motivation wasn't only in keeping the streak going, but mostly in making progress with my projects.

Now that I've reached the 365 day goal I've come to some conclusions:

  • Setting a goal helps to keep you going, but it doesn't necessarily has to be a streak of consecutive days commiting code. Josh mentioned this in his blog post as well, there are other metrics or goals you can aspire. Choose one that works for you and that is realistic.
  • A minimum of one commit per day worked for me, and usually I reached more (6,4 per day on average, with a maximum of 27 at some point). But it would be harder for me to set goal of, for example, 30 commits per week. It would put pressure on me every week to reach that goal and it would be counterproductive, for me at least.
  • Now that I've reached this 365 day goal I will keep the streak going as long as I'm comfortable with. As I mentioned, the one commit a day goal works for me as long as it doesn't interfere with my other commitments : I have a day job and regularly enjoy social, cultural and sports activities.
  • Next time I'm going away for a few days, being it a holiday or a climbing trip I will not take my laptop with me. It will break my streak, but I'm OK with that. Disconnecting from everything for a few days every once in a while is more important to me. Afterwards I'll probably pick it up again and start a new streak, but it doesn't matter how long it will be. I'm glad I reached the 365 days goal, but I don't necessary feel the need to repeat it.
  • As mentioned earlier, I found that my motivation lies in working on my projects, in seeing progress as I continue adding features and improving them.
  • I noticed a few similarities in motivation for doing sport (and other) activities. When you're running, for example, it helps to set a goal (a total distance or an average pace to reach) or a metric (once or twice a week, no matter how long the distance or time spent). Choose one that works for you to keep you motivated. But also allow yourself to take break if you need one and pick up later again. In the end you should enjoy the activity, setting a goal is just a way to keep you going, but the fun of doing the activity should drive you.
Many thanks to Josh to inspiring me to reach these conclusions, to all who supported me during the past year and good luck to everyone aspiring a goal, being it a commit streak or something else.

Kudos to those who still have a very long commit streak going!

Thursday, January 01, 2015

Happy 2015!

I wish you all an exciting 2015 in good health, with a lot of fun and big achievements in your projects! Good luck and joy to all my friends who are expecting a child this year!

Reviewing 2014


Well, 2014 was quite busy. Looking at my plans for 2014 a lot more happened than I expected : I learned Python when I started a new project, visited 7 capital cities and 3 continents, did a few climbing trips and got to visit San Francisco and Yosemite again :

  • celebrated New Year in Cebu, Philipines
  • visited Malaysia (Kuala Lumpur, Melakka, ...), Singapore and Hong Kong
  • bought a Google Nexus 5
  • attended FOSDEM 2014
  • resigned as a phpMyAdmin team member, creating some more time for other projects
  • finished a first blue route at Bleau
  • coorganised an info session about Google Summer of Code at Ghent University
  • started working on a tool to generate a trend graph of a build process, my first Python experience
  • attended Newline 0x04 and talked about reducing iptables configuration complexity
  • attended LinuxTag 2014 and talked about 'Reducing iptables configuration complexity' (presentation slides)
  • Finished a 10km race (Gentse stadsloop) in less than 55 minutes
  • Visited Maker Faire Paris : A lot of interesting stuff, mainly 3D printers, but some with a twist : pancake maker, an industrial welding robot mounted with an extruder, lots of hinges, bearings and other stuff printed ready to be used : no need to assemble
  • First time to visit Paris, I finally got to see the Eifel tower, Avenue des Champs Elysées, Arc de triomphe, Le Louvre (outside), Notre Dame, La Seine, Musée d'Orsay, Musée Rodin, Montmartre and Sacré Coeur
  • Released version 0.3 of Get Back GPS
  • Participated in a climbing training in L'Argentière-La Bessée and received a certificate for lead-climbing  and multi-pitch climbing (KVB3), had a lot of fun : climbing a lot and sending several 5b's and 5c's lead-climbing, some of them on sight.
  • Visited Amsterdam, the 6th capital city I went to this year
  • performed a lead-climber's fall at Klimax II
  • did a 55km cycling trip around Ghent
  • Donated blood for the first time
  • Released the first version of Buildtime Trend, a tool to create visual trends of  a software build process, written in Python and JavaScript
  • GetBack GPS had more than 10 contributors during 1 month, most of them are translating the app
  • Ran 16km/10mi for the first time
  • Spent a weekend in Fontainebleau with Maxime, Seba and Wolf, opening some routes.
  • Went to San Francisco to attend the GSoC 10th Anniversary Reunion and spent a few days in San Francisco and did a trip to Yosemite with Madhura Jayaratne. Shared a few beers with Justin and Josh of Keen.io, talking about Open Source commit streaks and Buildtime Trend.
  • Released GetBack GPS 0.4, introducing 7 more languages.
  • Spent a long weekend climbing in Fontainebleau with Peter, Dorinne, Cécile, Stef, Aline, Senne, Nik, Chris, Andreas, Luke and Barbara, climbing several yellow and blue routes.
  • and a second blood donation
  • finished a red climbing route
  • traveled to Lisbon.
Things to do in 2015:
  • Continue working on Buildtime Trend : offer it as a service (SaaS), add more stats, support more CI environments, ... 
  • Climb as much as possible, both indoor and outdoor, with trips to France during the Easter and summer holidays, and weekend trips to Fontainebleau or one of the climbing areas in the south of Belgium.
  • Visit Open Source and related conferences : FOSDEM, LinuxTag, EuroPython, DebConf and KeenCon.
I'm looking forward to where 2015 will bring me. Looking back at 2014 it is bound to be an exciting year again.