Friday, May 31, 2013

Testing Chaos with Automated Configuration Management solutions

No noise making.

But let's be real, think of the count of community contributed (or mysterious closed-and-sold 3rd Party) services, frameworks, library and modules put to use for managing your ultra-cool self-healing self-reliant scalable Infrastructure requirements. Now with so many cogs collaborating in the infra-machine, a check on their collaboration seems rather mandatory like any other integration test for your in-house managed service. 
After all that was key idea behind having automated configuration management itself.

Now the utilities like Puppet/Chef have been out there accepted and used by dev & ops folks for quite some time now.
But the issue with the initially seen amateur testing styles is it evolved from the non-matching frame of 'Product' oriented unit/integration/performance testing. 'Product' oriented testing focus more on what happens inside the coded logic and less on how user gets affected by product.
Most of the initial tools released for testing logic developed in Chef/Puppet were RSpec/Cucumber inspired Product testing pieces. Now for the major part of installing a package, restarting a service or pushing artifacts these tests are almost non-required as the main functionality for per-say installing package_abc is already tested inside the framework being used.
So coding to "ask" to install package_abc and testing if it has been asked seems futile.

That's the shift. The logic developed for Infrastructure acts as a glue to all other applications created in house and 3rd party. Here in Infrastructure feature development there is more to test for the effect it has on the it's users (software/hardware) and less on internal changes (dependencies and dynamic content). Now the stuff in parentheses here means a lot more than seems... let's get into detail of it.

Real usability of Testing is based on keeping sanctity of WHAT needs to be tested WHERE.

Software/Hardware services that collaborate with the help of Automated Infrastructure logic needs major focus of testing. These services can be varying from the
  • in-house 'Product', that is the central component you are developing
  • 3rd Party services it collaborates with,
  • external services it utilizes for what it doesn't host,
  • operating system that it supports and Ops-knows what not.

Internal changes mainly revolve around
  • Resources/Dependencies getting called in right order and grouped for specific state.
  • It also relates to correct generation/purging of dynamic content, that content can itself range as
    • non-corrupt configuration files generated of a template
    • format of sent configuration data from one Infra-component to another for reflected changes
    • dynamically creating/destroying service instances in case of auto-scalable infrastructure

One can decide HOW, on ease and efficiency basis.

Unit Tests work for the major portion of 'Internal Changes' mentioned before using chefspecrspec-chef, rspec-puppet like libraries are good enough. They can very well test the dependency order and grouping management as well as the different data effect on non-corrupt configuration generation from templates.

Integration Tests in this perspective are a of a bit interesting and evolutionary nature. Here we have to ensure the "glue" functionality we talked about for Software/Hardware service is working properly. These will confirm that every type of required machine role/state can be achieved flawlessly, call them 'State Generation Test'. They also need to confirm the 'Reflected Changes Test' across Infra-component as mentioned in Internal changes.
Now utilities like test-kitchen/docker in collaboration with vagrant, docker, etc. help placing them in your Continuous Integration pipeline. This would even help in testing same service across multiple linux distros if that's the plan to support.
Library 'ServerSpec' is also a little nifty piece to write quick final state check scripts.
Then final set of Integration Testing is implemented in form of Monitoring on your all managed/affecting Infrastructure components. This is the final and ever-running Integration Test.

Performance Tests, yes even they are required for it. Tools like ChaosMonkey enable you to enable your Infra to be self-healing and auto-scalable. Should be load-test noticing dynamic containers count and behavior if auto-scalability is a desired functionality too.


  1. It is truly a great and helpful piece of information.
    I am satisfied that you simply shared this useful information with us.
    Please stay us informed like this. Thanks for sharing. I know something information, to know you can click here
    Test and Tag brisbane
    rcd testing services brisbane

  2. We are a leading software development company,which works as per the client requirements and give provide software.

  3. Test and Tag can keep your electrical appliances in the place of work compliant with government legislation.
    Be at liberty to Speak to me at any time for virtually any queries or simply a no obligation free quote.

    Test and Tag
    Test and Tag brisbane

  4. Test and Tag Brisbane is committed to providing appliance safety solutions which delivering the highest level of quality services.
    We offer a large range of Independent Test And Tag.
    This is a authorized requirement that all these types of appliances are tested consistently in order that These are electrically Risk-free for use.
    you can try this out,

    Test and Tag brisbane

  5. This is an awesome post.Really very informative and creative contents. These concept is a good way to enhance the knowledge.I like it and help me to development very well.Thank you for this brief explanation and very nice information.
    Cloud Computing Training in Chennai

  6. BlueHost is ultimately one of the best web-hosting provider with plans for all of your hosting needs.

  7. Very useful and informative content has been shared out here, Thanks for sharing it.
    Visit Learn Digital Academy for more information on Digital marketing course in Bangalore

  8. Nice blog, this blog provide the more information. Thank you so much for sharing with us and thanks people who give me this useful information.

    Brand Development Company | Travel Software Development Company | Software Development Solutions in India | Dynamic Websites Design Services | Mobile App Development Company

  9. Thank you for the nice article here. Really nice and keep update to explore more gaming tips and ideas.

    Game Testing Services

    Video Game Tester

    IOS Game QA Tester

    Game Security Testing

    Game QA

  10. Thank you for the nice article here. Really nice and keep update to explore more gaming tips and ideas.

    PS4 Game Tester

    Game Testing Services

    Video Game Testing Companies


  11. read this above post its very greatful for me thanks sharing this post ,great post.
    Best Ice Fishing Gloves Best Ice Fishing Gloves Best Ice Fishing Gloves

  12. Thanks for the information..

    We have been passionate about crafting iconic Eurasian circuits specially Caucasus (Armenia, Azerbaijan & Georgia), Central Asia (Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan and Uzbekistan) and Eastern Europe (Russia, Serbia, and Ukraine) for more than 8 years.

    Adventure Tour Packages | Ukraine Tour Package | Trekking Tour Packages | Eastern Europe Tours| Silk Route Tour Packages| Russia Tour Package | Almaty Tour Package | Tashkent Group Packages

  13. What an exquisite post you have written! He would have come to relieve beginners like me! This could be very clear, precise and descriptive really! Thank you very much, author. Keep writing for us like this :)

  14. I was very interested in the article , it’s quite inspiring I should admit. I like visiting your site since I always come across interesting articles like this one. Keep sharing! Regards. Read more about
    Offshore software testing services
    software testing services company
    software testing services
    Software Qa Services
    quality assurance service providers
    Performance testing services
    Security testing services
    software testing Companies