tag:blogger.com,1999:blog-71540531144394005562024-03-13T00:08:37.989-07:00just few tutsjust few HOW TO tutorials in complete LAY MAN versionabionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.comBlogger42125tag:blogger.com,1999:blog-7154053114439400556.post-17073386340528453462014-10-22T08:54:00.000-07:002014-10-22T08:54:17.772-07:00What qualifies a Project to be a Product?<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr">
<span style="color: #e06666;"><b>Disclaimer:</b><br /><i>This post is based upon a piece of dream (not fiction). Any relation to any enterprise grade product is purely coincidental.</i></span></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
---</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Now, <b><span style="color: #e06666;">this is not a flame post against Proprietary Software</span></b>. I'm a FOSS supporter but with understanding that <b>for some Business... </b></div>
<div dir="ltr">
<b>* paid 24x7 support is a lot more critical than quality</b></div>
<div dir="ltr">
<b>* need to put trust in a product where the Vendor is bound by agreement to help</b></div>
<div dir="ltr">
<b>which is perfectly fine.</b></div>
<div dir="ltr">
Depending on a business requirement and policies, there need to be different solutions provided. Some community supported FOSS and then many large corporations (backed for years) Enterprise Software.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Again, no this post is not on license of software.</div>
<b><span style="color: #e06666;">This post is on some core values of a software (FOSS or Proprietary) that make it eligible to be used at a worthy scale in a Organization that is depending on it.</span></b><br />
<div dir="ltr">
These also stand true for any user of that Software Project, but are more crucial for users depending their economy on it.</div>
<div dir="ltr">
And are ethically right of the Corporations which are paying big bucks for a piece of code sold to them on high brand and big promises.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Here on I use "Products" for all paid-for (some Enterprise grade) Projects that I got to work on since college and experience the truth underneath.</div>
<div dir="ltr">
---</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
So... <b><span style="font-size: large;">What qualifies a Project to be a Product?</span></b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
</div>
<ul style="text-align: left;">
<li><b>2 old-skool fundamental mantras: loose coupling; high cohesion</b>There are "Products" with bunch of modules that have separate responsibilities. Correct approach. But when you start to build around them, sometimes they are not that independent in containing self responsibilities.<br />That's worse than not having your project modular at all, at least that way you promote users to treat it as a black-box.<br />"Product" need to be well modular-ized emitting the age old beautiful development practices of keeping your modules loosely coupled, strongly cohesive.</li>
</ul>
<ul style="text-align: left;">
<li><b>isolated from client specific details</b><br />Anything and everything that depicts details about client specific implementation need to be managed as a configuration provided explicitly. There shouldn't be any necessity of find-replace any hard-coded "config text" in source or set-up files for the "Product". These details shan't be required to be packaged with custom traditional set-up of "Product".</li>
</ul>
<ul style="text-align: left;">
<li><b>generic out-of-the-box setup... tested and isolated</b><br />The "Product" installer binaries shall be self-contained. They can be O.S. distribution specific, which is fine.<br />They should be unaffected of O.S. level restrictions that may be there or not. Say if SELinux is enabled, your setup shall be able to initiate changes required for the "Product" to function.<br />Any dependency shall be either bundled in the installer package, or shall utilize the targeted package manager to lay it down for itself.<br />The installer shall be tested completely in the mode it is supposed to be used. For example, if it sets up the machine remotely then it shall be capable of handling all tasks remotely by itself and not depend on user to place some files for it first or in-between.</li>
</ul>
<ul style="text-align: left;">
<li><b>no mesh between services required for initiation</b><br />The "Product" might have distributed architecture support, might have modular components collaborating.<br />Now these distributed components need to be aware of each other to be able to collaborate. The components shall be robust enough to handle unavailable dependency components and regain activity once available.<br />This awareness can be maintained on a component dedicated to dynamic configuration query and update, where all component instances export details about self and gather information on others. Environment preparation mechanism can populate the initial dynamic information which on requirement can be updated in collaboration-enabling component and gathered by others.<br />Every component can persist information required for it within itself as well, but that shouldn't be dependent on the other components. If component-F collaborates with component-A and requires component-A to mark some activities for it even to start itself. Bad design.</li>
</ul>
<ul style="text-align: left;">
<li><b>don't promote deprecated technology in newer component</b><br />Some "Product" have big lifetime, depending on their vastness &/or critical nature. This might lead to existence of certain obsolete technologies (like SOAP in yr2014) usage in newer components.<br />You shouldn't start rebuilding entire perfectly working "Product" for that, yes. But neither shall you use that as an excuse for holding back entire new development around it. It will cripple the "Product" much faster of surviving in newer circumstances if it can't use the power required for them.<br />Build a mid-layer contract API and seclude your new work over your legacy "Product". Then build new features over that mid-layer API.<br />This will help you avoid "Product" becoming a bloatware just because new features can't be used inherently from the design itself. It will also help contain the sanity of perfectly working "Product" from some (library, etc.) changes required for new features. And will let you use more advance current best practices for all the newer work, not cripple it.<br /><i>If not "Product" in entirety; at least every individual component in itself shall follow unified design/development/interface/platform strategy... it shouldn't have bifurcation of ideals, if necessary a new component shall be carved out and plugged-in instead of corrupting the existing piece.</i></li>
</ul>
<ul style="text-align: left;">
<li><b>different style of code, different degree of documentation</b><br />If your code-base is not very huge, beautifully written clean and modular code can survive without documentation.<br />If your code is not clean (don't judge it yourself, ask someone expert in language/framework but unaware of logic to guess) then have freaking documentation all over it.<br />If your code-base is "really" huge, even if your code is mostly clean... have a basic module level documentation at least.<br />So anyone using it knows how to handle them and build upon or around them. Anyone visiting the code-base many years after (if you think your "Product" is capable) with lot improved language features is still able to make sense of what was carved with stone on cave walls.</li>
</ul>
<ul style="text-align: left;">
<li><b>building upon/around it shouldn't involve hack-y ways</b><br />In small home-brewed solutions, unpack-replace-repack is still arguably accepted solution.<br />In a freaking "Product", it shall expose an elegant yet secure API interface to extend features. If it allows overriding of existing features, even they shall be plugged in via that interface causing the built-in feature to be subdued.<br />It's not effortless but is a sane and secure method. The "Product" need this embedded in it's design.</li>
</ul>
<ul style="text-align: left;">
<li><b>for all kinds of data-set involved, should have a proper data-modification strategy</b><br />Hand modification seem harmless at development stage, even at testing sometimes. In production if any kind of data modification done during setup or updates doesn't have a migration/rollback strategy around it, it's a danger zone signal.<br />Like proper DB migration scripts over (and not plain diffs of) current version of scripts. I might seem obvious as all other points, but there are bunch of Product missing one or other variations of it.</li>
</ul>
<ul style="text-align: left;">
<li><b>take care of sensitive data involved even during setup of "Product"</b><br />If your Product in any manner forces display or storage of sensitive data (usernames, passwords, machine names, network details, yada yada yada), danger zone again.</li>
</ul>
<ul style="text-align: left;">
<li><b>have at least some level of sanity tests for all logic flows</b><br />Even in this age of software development, if importance of software testing (acceptance, regression, security) need to be advocated to you, it's disaster.<br />To state obvious... any change can be tracked properly, all technologists have some level of idea and trust on what's where doing what, people picking it much later can build over it without breaking anything.</li>
</ul>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
and some things, but that for some other time.....</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
---</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<b>LONG QUESTION</b>:</div>
<div dir="ltr">
Could a piece of software sold by "so called tech moghul" Organizations be so crappy that the only purpose it fills is vendor lock-in for years of licensing and not even the creators know it's buggy to its core?</div>
<div dir="ltr">
<b>SHORT ANSWER</b>:</div>
<div dir="ltr">
Yes! So chose with care.</div>
</div>
abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com15tag:blogger.com,1999:blog-7154053114439400556.post-69340262278267944122014-09-18T05:40:00.001-07:002014-09-22T02:55:29.583-07:00base checklist: 10 points to decide if chose an OSS for Production or not<div dir="ltr" style="text-align: left;" trbidi="on">
One question you get from skeptics (who are actually really important for quality check) while discussion on picking an OpenSource solution instead of a support attached closed source one. Which is how to trust it to be safe for Production release.<br><br>The question actually even suits which OSS to pick, when there are several.<br><br>Question we are trying to answer here is... <b>How to pick an OpenSource solution that will live long and prosper, not turn into a rot that smells on any change in Production on updates over period of time.</b><br><br>First just to mention again that almost every Technologist already understands. There is no guarantee just "ensured" support over closed source software to guarantee it's safety or supporting future technical growth. I don't wanna dwell into the dangers that it brings in, 'cuz this is not the post for that. That's entirely other exhausted list.</div><div dir="ltr" style="text-align: left;" trbidi="on"><br>
<div>
<i><b>So on what to see in OpenSource software that helps you decide it is trusted to be included for production release...</b></i></div>
<div>
<br></div><div><br></div><div><span style="font-family: sans-serif;">While weighing in for inclusion of any big or small OpenSource utility into your Production list, following checklist shall help:</span><br></div><div><span style="font-family: sans-serif;"><br></span></div>
<div>
1*) OSS have Licenses too</div><div>First of all check if their License suits inclusion with licensing of your project. Example, People have been seeking ways (somewhat succeded) to get ZFS on GNU/Linux.</div><div><br></div><div>2*) Is project active "enough"</div><div>Second quick check is seeing if project has been inactive for a dangerous period. Now for every kind of project, a dangerous period differs widely. Would have to depend on better judgment of self and trusted community you know. Like for a library providing certain algorithm, post stable release changes would be a lot slower. But for a webdev framework, with current tradition... it'll be popping new minor releases now and then.</div><div><br></div><div>Now few things for which you'll need to read around a little....<br><span style="font-family: sans-serif;">Sources to recon about following attributes: Mailing lists, Issue boards, IRC, Twitter streams, may be others depending on project</span></div><div><font face="sans-serif"><br></font>3*) How much active and inclusive is its community</div><div><span style="font-family: sans-serif;">How well do they handle PullRequests and Issues raised on their project. This includes the readiness on response and adapting a better direction, both but mainly former.</span></div><div><span style="font-family: sans-serif;">How well they handle risks and vulnerabilities reported, if any. Quickest patch is not the main measure, most important is accepting it and providing a workaround till main issue gets resolved.</span></div><div><br></div><div>4*) Good core team matters<span style="font-family: sans-serif;"> </span><span style="font-family: sans-serif;">(they need not be very popular)</span></div><div>Check who forms core team maintaining that OSS. Some other projects of their, even if not popular would give you an idea on how much and how well they maintain their projects. </div><div><br>5*) If Industry already loves it</div><div>Not a litmus test though strengthens community support and quality check.</div><div>Look for who all in Industry is already using it mainstream, also if you like the softwares they have developed. Just shoot a tweet/mail to them... people are mostly helpful. Don't give up on humanity. ;)</div><div><br>6*) Need to scan it personally anyhow</div><div>Try it in a sandbox first, monitor it's not spawning requests to domains it's not supposed to. Not creating any suspicious behavior you don't expect from it.</div><div>Also, it survives your production security lockdown, not all projects behave same under restrictions.<br><br>7*) Send it on a marathon</div><div>Put it under performance test yourselves. There might be preexisting load test results available, and might be accurate as well. But not all implementations suit all projects. Check it under PoC of your implementation behavior with expected concurrency and latency.<br><br>8*) Does it tailor fit</div><div>If it actually provides what you desire without putting a hack around, give it a chance. If not so, confirm that it suits the design and wouldn't break with project philosophy from maintainers over the coming recent versions at least.</div><div><br></div><div>9*) How easy is to resolve an issue</div><div>Is project community/developers active enough to help guide around any problems faced.</div><div><br></div><div>10*) Do you love supporting FOSS</div><div>If yes, welcome to the world of awesomeness. Some mediocrity (not below that, then look something else) at some of points above would only drive you strengthen the project. It's opensource, at least technologists are not supposed to live with the problem if faced.</div><div><br></div>
</div>
abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com2tag:blogger.com,1999:blog-7154053114439400556.post-29739075369913862982014-02-03T13:42:00.001-08:002014-02-15T14:03:13.895-08:00golang ~ get local changes into GOPATH without pushing them upstream<div dir="ltr" style="text-align: left;" trbidi="on">
To get your local Golang repo's sym-linked at your GOPATH and local changes available...<br />
<span style="font-family: Times, Times New Roman, serif;"><br /></span>
<pre style="white-space: pre-wrap; word-wrap: break-word;"><span style="font-family: Times, Times New Roman, serif;">goenv_link(){
if [ $# -ne 2 ]; then
echo "Links up current dir to it's go-get location in GOPATH"
echo "SYNTAX: goenv_linkme <local-repo-path> <path-provided-go-get>"
return 1
fi
_REPO_DIR=$1
_REPO_URL=$2
_TMP_PWD=$PWD
cd $_REPO_DIR
if [ -d "${GOPATH}/src/${_REPO_URL}" ]; then
echo "$_REPO_URL already exists at GOPATH $GOPATH"
go get "${_REPO_URL}"
return 1
fi
_REPO_BASEDIR=$(dirname "${GOPATH}/src/${_REPO_URL}")
if [ ! -d "${_REPO_BASEDIR}" ]; then
mkdir -p "${_REPO_BASEDIR}/src"
fi
ln -sf "${PWD}" "${GOPATH}/src/${_REPO_URL}"
go get "${_REPO_URL}"
cd $_TMP_PWD
}
alias goenv_linkme="goenv_link $PWD"</path-provided-go-get></local-repo-path></span></pre>
<br />
---<br />
<br />
<br />
Every now and then working on my favorite new programming language Golang, I have inter-dependent changes among different packages. To confirm their as-required working state, I'd like the GOPATH to provide the compiled object with local-changes included.<br />
<br />
The utility I've been using to push local package changes to GOPATH provided object is following "goenv_alpha" bash function as a shell-profile provided utility.<br />
<br />
Say, I've a golang project "github.com/abhishekkr/goshare" which utilizes "github.com/abhishekkr/goshare/httpd", "github.com/abhishekkr/goshare/zeromq" and few more.<br />
<br />
If I make some local changes at "{PROJECTS}/goshare" and "{PROJECTS}/goshare/httpd". To push those into GOPATH provided package for testing, following commands using below function "goenv_alpha" shell-util would do the job...<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: Times, Times New Roman, serif;">$ goenv_alpha "{PROJECTS}/goshare" "github.com/abhishekkr/goshare"<br />$ goenv_alpha "{PR..}/goshare/httpd" "github.com/abhishekkr/goshare/httpd</span>"</blockquote>
<div>
<br /></div>
<div>
These commands will ask you to make a backup file for current existing version of package resource from GOPATH, you can give any name... which will be asked while restoring or you can leave it empty to avoid creating a backup file.<br />
<br />
~<br />
<br />
<pre style="box-sizing: border-box; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; line-height: 18px;"><div class="line" id="LC65" style="box-sizing: border-box; padding-left: 10px;">
<span style="color: #333333;">goenv_alpha</span><span class="o" style="box-sizing: border-box; color: #333333;"><b>(){
</b></span><span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">_TMP_PWD</span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">=</span><span class="nv" style="box-sizing: border-box;"><span style="color: teal;">$PWD
</span></span><span style="color: #333333;"> </span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">if</span><span style="color: #333333;"> </span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">[</span><span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">$# </span><span style="color: #333333;">-ne 2 </span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">]</span><span class="p" style="box-sizing: border-box; color: #333333;">;</span><span style="color: #333333;"> </span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">then
</span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;"> </span><span class="nb" style="box-sizing: border-box; color: #0086b3;">echo</span><span style="color: #333333;"> </span><span class="s2" style="box-sizing: border-box;"><span style="color: #dd1144;">"Provide Alpha changes usable as any other go package."
</span></span><span style="color: #333333;"> </span><span class="nb" style="box-sizing: border-box; color: #0086b3;">echo</span><span style="color: #333333;"> </span><span class="s2" style="box-sizing: border-box;"><span style="color: #dd1144;">"Just the import path changes to 'alpha/<project-name>'"
</project-name></span></span><span style="color: #333333;"> </span><span class="nb" style="box-sizing: border-box; color: #0086b3;">echo</span><span style="color: #333333;"> </span><span class="s2" style="box-sizing: border-box; color: #dd1144;">"SYNTAX: goenv_alpha <path-to-project-dir-with-alpha-changes> <go-get-import-path-for-it>"
</go-get-import-path-for-it></path-to-project-dir-with-alpha-changes></span><span style="color: #333333;"> </span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">return </span><span style="color: #333333;">1
</span><span style="color: #333333;"> </span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">fi
</span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">_REPO_DIR</span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">=</span><span class="nv" style="box-sizing: border-box; color: teal;">$1
</span><span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">_REPO_URL</span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">=</span><span class="nv" style="box-sizing: border-box;"><span style="color: teal;">$2
</span><span style="color: #333333;"> </span><span class="nb" style="box-sizing: border-box; color: #0086b3;">cd</span><span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">$_REPO_DIR
</span></span><span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">_PKG_PARENT_NAME</span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">=</span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">$(</span><span style="color: #333333;">dirname </span><span class="nv" style="box-sizing: border-box; color: teal;">$PWD</span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">)
</span><span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">_PKG_NAME</span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">=</span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">$(</span><span style="color: #333333;">basename </span><span class="nv" style="box-sizing: border-box; color: teal;">$PWD</span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">)
</span></div>
<div class="line" id="LC80" style="box-sizing: border-box; padding-left: 10px;">
<span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">_PKG_NAME_IN_REPO</span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">=</span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">$(</span><span style="color: #333333;">basename </span><span class="nv" style="box-sizing: border-box; color: teal;">$_REPO_URL</span><span class="k" style="box-sizing: border-box; color: #333333;"><b>)
</b></span><span style="color: #333333;"> </span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">if</span><span style="color: #333333;"> </span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">[</span><span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">$_PKG_NAME_IN_REPO</span><span style="color: #333333;"> !</span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">=</span><span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">$_PKG_NAME</span><span style="color: #333333;"> </span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">]</span><span class="p" style="box-sizing: border-box; color: #333333;">;</span><span style="color: #333333;"> </span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">then
</span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;"> </span><span class="nb" style="box-sizing: border-box; color: #0086b3;">echo</span><span style="color: #333333;"> </span><span class="s2" style="box-sizing: border-box; color: #dd1144;">"Path for creating alpha doesn't match the import 'url' for it."
</span><span style="color: #333333;"> </span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">return </span><span style="color: #333333;">1
</span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">fi
</span><span style="color: #333333;">
</span><span class="sb" style="box-sizing: border-box; color: #dd1144;">`</span><span style="color: #333333;">go build -work . 2> /tmp/</span><span class="nv" style="box-sizing: border-box; color: teal;">$_PKG_NAME</span><span class="sb" style="box-sizing: border-box; color: #dd1144;">`
</span><span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">_BUILD_PATH</span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">=</span><span class="sb" style="box-sizing: border-box; color: #dd1144;">`</span><span style="color: #333333;">cat /tmp/</span><span class="nv" style="box-sizing: border-box; color: teal;">$_PKG_NAME</span><span style="color: #333333;"> </span><span class="p" style="box-sizing: border-box; color: #333333;">|</span><span style="color: #333333;"> sed </span><span class="s1" style="box-sizing: border-box; color: #dd1144;">'s/WORK=//'</span><span class="sb" style="box-sizing: border-box; color: #dd1144;">`
</span><span style="color: #333333;"> </span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">if</span><span style="color: #333333;"> </span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">[</span><span style="color: #333333;"> ! -d </span><span class="nv" style="box-sizing: border-box; color: teal;">$_BUILD_PATH</span><span style="color: #333333;"> </span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">]</span><span class="p" style="box-sizing: border-box; color: #333333;">;</span><span style="color: #333333;"> </span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">then
</span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;"> </span><span class="nb" style="box-sizing: border-box; color: #0086b3;">echo</span><span style="color: #333333;"> </span><span class="s2" style="box-sizing: border-box; color: #dd1144;">"An error occured while building, it's recorded at /tmp/$_PKG_NAME"
</span><span style="color: #333333;"> </span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">return </span><span style="color: #333333;">1
</span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">fi
</span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;"> </span><span style="color: #333333;">rm -f /tmp/</span><span class="nv" style="box-sizing: border-box;"><span style="color: teal;">$_PKG_NAME
</span><span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">_CURRENT_OBJECT_PATH</span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">=</span><span class="s2" style="box-sizing: border-box; color: #dd1144;">"${GOPATH}/pkg/${GOOS}_${GOARCH}"
</span></span><span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">_CURRENT_OBJECT</span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">=</span><span class="s2" style="box-sizing: border-box; color: #dd1144;">"${_CURRENT_OBJECT_PATH}/${_REPO_URL}.a"
</span><span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">_NEW_OBJECT</span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">=</span><span class="s2" style="box-sizing: border-box;"><span style="color: #dd1144;">"${_BUILD_PATH}/_${_PKG_PARENT_NAME}/${_PKG_NAME}.a"
</span><span style="color: #333333;"> </span><span class="nb" style="box-sizing: border-box; color: #0086b3;">echo</span><span style="color: #333333;"> </span><span class="s2" style="box-sizing: border-box; color: #dd1144;">"Do you wanna backup current object? If yes enter a filename for it: "
</span></span><span style="color: #333333;"> </span><span class="nb" style="box-sizing: border-box; color: #0086b3;">read </span><span style="color: #333333;">GO_ALPHA_BACKUP
</span><span style="color: #333333;"> </span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">if</span><span style="color: #333333;"> </span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">[</span><span style="color: #333333;"> ! -z </span><span class="nv" style="box-sizing: border-box; color: teal;">$GO_ALPHA_BACKUP</span><span style="color: #333333;"> </span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">]</span><span class="p" style="box-sizing: border-box; color: #333333;">;</span><span style="color: #333333;"> </span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">then
</span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;"> </span><span style="color: #333333;">mv </span><span class="nv" style="box-sizing: border-box; color: teal;">$_CURRENT_OBJECT</span><span style="color: #333333;"> </span><span class="s2" style="box-sizing: border-box; color: #dd1144;">"${_CURRENT_OBJECT_PATH}/${_REPO_URL}/${GO_ALPHA_BACKUP}.backup"
</span><span style="color: #333333;"> </span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">fi
</span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;"> </span><span style="color: #333333;">mv </span><span class="nv" style="box-sizing: border-box; color: teal;">$_NEW_OBJECT</span><span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">$_CURRENT_OBJECT
</span><span style="color: teal;">
</span><span style="color: #333333;"> </span><span class="nb" style="box-sizing: border-box; color: #0086b3;">cd</span><span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">$_TMP_PWD
</span><span style="color: #333333;"> </span><span class="nb" style="box-sizing: border-box; color: #0086b3;">echo</span><span style="color: #333333;"> </span><span class="s2" style="box-sizing: border-box; color: #dd1144;">"\nAlpha changes have been updated at ${_CURRENT_OBJECT}."
</span><span style="color: #333333; font-weight: bold;">}</span></div>
</pre>
<br />
~<br />
<br />
You can undo the pushing of local changes inclusive package resource if you have created a backup file for earlier existing file.<br />
<br />
Following commands utilizes the below provided shell-util function "goenv_alpha_undo".<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: Times, Times New Roman, serif;">$ goenv_alpha_undo "{PROJECTS}/goshare" "github.com/abhishekkr/goshare"<br />$ goenv_alpha_undo "{PR..}/goshare/httpd" "github.com/abhishekkr/goshare/httpd"</span></blockquote>
<br />
This will list you the names of backup files present if any, then you can provide the name of your chosen backup file and restore to that package state.<br />
<br />
~<br />
<pre style="box-sizing: border-box; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; line-height: 18px; text-align: left;"><span style="color: #333333;">goenv_alpha_undo</span><span class="o" style="box-sizing: border-box; color: #333333;"><b>(){
</b></span><span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">_TMP_PWD</span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">=</span><span class="nv" style="box-sizing: border-box;"><span style="color: teal;">$PWD
</span></span><span style="color: #333333;"> </span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">if</span><span style="color: #333333;"> </span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">[</span><span style="color: #333333;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">$# </span><span style="color: #333333;">-ne 2 </span><span class="o" style="box-sizing: border-box; color: #333333; font-weight: bold;">]</span><span class="p" style="box-sizing: border-box; color: #333333;">;</span><span style="color: #333333;"> </span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">then
</span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;"> </span><span class="nb" style="box-sizing: border-box; color: #0086b3;">echo</span><span style="color: #333333;"> </span><span class="s2" style="box-sizing: border-box; color: #dd1144;">"Provide Alpha changes usable as any other go package."
</span><span style="color: #333333;"> </span><span class="nb" style="box-sizing: border-box; color: #0086b3;">echo</span><span style="color: #333333;"> </span><span class="s2" style="box-sizing: border-box; color: #dd1144;">"Just the import path changes to 'alpha/<project-name>'"
</project-name></span><span style="color: #333333;"> </span><span class="nb" style="box-sizing: border-box; color: #0086b3;">echo</span><span style="color: #333333;"> </span><span class="s2" style="box-sizing: border-box; color: #dd1144;">"SYNTAX: goenv_alpha <path-to-project-dir-with-alpha-changes> <go-get-import-path-for-it>"
</go-get-import-path-for-it></path-to-project-dir-with-alpha-changes></span><span style="color: #333333;"> </span><span class="k" style="box-sizing: border-box; color: #333333; font-weight: bold;">return </span><span style="color: #333333;">1
</span><div style="color: #333333;">
<span class="k" style="box-sizing: border-box; font-weight: bold;">fi
</span><span class="k" style="box-sizing: border-box; font-weight: bold;"> </span><span class="nv" style="box-sizing: border-box; color: teal;">_REPO_DIR</span><span class="o" style="box-sizing: border-box; font-weight: bold;">=</span><span class="nv" style="box-sizing: border-box; color: teal;">$1</span><br />
<span class="nv" style="box-sizing: border-box; color: teal;">_REPO_URL</span><span class="o" style="box-sizing: border-box; font-weight: bold;">=</span><span class="nv" style="box-sizing: border-box; color: teal;">$2
</span><span class="nv" style="box-sizing: border-box; color: teal;">
</span> <span class="nb" style="box-sizing: border-box; color: #0086b3;">cd</span> <span class="nv" style="box-sizing: border-box; color: teal;">$_REPO_DIR
</span> <span class="nv" style="box-sizing: border-box; color: teal;">_PKG_PARENT_NAME</span><span class="o" style="box-sizing: border-box; font-weight: bold;">=</span><span class="k" style="box-sizing: border-box; font-weight: bold;">$(</span>dirname <span class="nv" style="box-sizing: border-box; color: teal;">$PWD</span><span class="k" style="box-sizing: border-box; font-weight: bold;">)
</span> <span class="nv" style="box-sizing: border-box; color: teal;">_PKG_NAME</span><span class="o" style="box-sizing: border-box; font-weight: bold;">=</span><span class="k" style="box-sizing: border-box; font-weight: bold;">$(</span>basename <span class="nv" style="box-sizing: border-box; color: teal;">$PWD</span><span class="k" style="box-sizing: border-box; font-weight: bold;">)
</span>
<span class="nv" style="box-sizing: border-box; color: teal;">_PKG_NAME_IN_REPO</span><span class="o" style="box-sizing: border-box; font-weight: bold;">=</span><span class="k" style="box-sizing: border-box; font-weight: bold;">$(</span>basename <span class="nv" style="box-sizing: border-box; color: teal;">$_REPO_URL</span><span class="k" style="box-sizing: border-box; font-weight: bold;">)
</span> <span class="k" style="box-sizing: border-box; font-weight: bold;">if</span> <span class="o" style="box-sizing: border-box; font-weight: bold;">[</span> <span class="nv" style="box-sizing: border-box; color: teal;">$_PKG_NAME_IN_REPO</span> !<span class="o" style="box-sizing: border-box; font-weight: bold;">=</span> <span class="nv" style="box-sizing: border-box; color: teal;">$_PKG_NAME</span> <span class="o" style="box-sizing: border-box; font-weight: bold;">]</span><span class="p" style="box-sizing: border-box;">;</span> <span class="k" style="box-sizing: border-box; font-weight: bold;">then
</span><span class="k" style="box-sizing: border-box; font-weight: bold;"> </span><span class="nb" style="box-sizing: border-box; color: #0086b3;">echo</span> <span class="s2" style="box-sizing: border-box; color: #dd1144;">"Path for creating alpha doesn't match the import 'url' for it."
</span> <span class="k" style="box-sizing: border-box; font-weight: bold;">return </span>1
<span class="k" style="box-sizing: border-box; font-weight: bold;">fi
</span><span class="k" style="box-sizing: border-box;">
<b> </b></span><span class="nv" style="box-sizing: border-box; color: teal;">_CURRENT_OBJECT_PATH</span><span class="o" style="box-sizing: border-box; font-weight: bold;">=</span><span class="s2" style="box-sizing: border-box; color: #dd1144;">"${GOPATH}/pkg/${GOOS}_${GOARCH}"
</span> <span class="nv" style="box-sizing: border-box; color: teal;">_CURRENT_OBJECT</span><span class="o" style="box-sizing: border-box; font-weight: bold;">=</span><span class="s2" style="box-sizing: border-box; color: #dd1144;">"${_CURRENT_OBJECT_PATH}/${_REPO_URL}.a"
</span> <span class="nv" style="box-sizing: border-box; color: teal;">_BACKUP_OBJECT</span><span class="o" style="box-sizing: border-box; font-weight: bold;">=</span><span class="s2" style="box-sizing: border-box; color: #dd1144;">"${_BUILD_PATH}/_${_PKG_PARENT_NAME}/${_PKG_NAME}.a"
</span> <span class="nb" style="box-sizing: border-box; color: #0086b3;">echo</span> <span class="s2" style="box-sizing: border-box; color: #dd1144;">"Available package files are:"
</span> ls -1 <span class="nv" style="box-sizing: border-box; color: teal;">$_CURRENT_OBJECT_PATH</span>/<span class="nv" style="box-sizing: border-box; color: teal;">$_REPO_URL</span> <span class="p" style="box-sizing: border-box;">|</span> grep <span class="nv" style="box-sizing: border-box; color: teal;">$_PKG_NAME</span> <span class="p" style="box-sizing: border-box;">|</span> grep -v grep
<span class="nb" style="box-sizing: border-box; color: #0086b3;">echo</span> <span class="s2" style="box-sizing: border-box; color: #dd1144;">"Enter your backup filename for it: "
</span> <span class="nb" style="box-sizing: border-box; color: #0086b3;">read </span>GO_ALPHA_BACKUP
<span class="k" style="box-sizing: border-box; font-weight: bold;">if</span> <span class="o" style="box-sizing: border-box; font-weight: bold;">[</span> -z <span class="nv" style="box-sizing: border-box; color: teal;">$GO_ALPHA_BACKUP</span> <span class="o" style="box-sizing: border-box; font-weight: bold;">]</span><span class="p" style="box-sizing: border-box;">;</span> <span class="k" style="box-sizing: border-box; font-weight: bold;">then
</span><span class="k" style="box-sizing: border-box; font-weight: bold;"> </span><span class="nb" style="box-sizing: border-box; color: #0086b3;">echo</span> <span class="s2" style="box-sizing: border-box; color: #dd1144;">"\nNo Backup file was entered."</span> <span class="p" style="box-sizing: border-box;">;</span> <span class="k" style="box-sizing: border-box; font-weight: bold;">return </span>1
<span class="k" style="box-sizing: border-box; font-weight: bold;">fi
</span><span class="k" style="box-sizing: border-box; font-weight: bold;"> </span>mv <span class="s2" style="box-sizing: border-box; color: #dd1144;">"${_CURRENT_OBJECT_PATH}/${_REPO_URL}/${GO_ALPHA_BACKUP}"</span> <span class="nv" style="box-sizing: border-box; color: teal;">$_CURRENT_OBJECT
</span>
<span class="nb" style="box-sizing: border-box; color: #0086b3;">cd</span> <span class="nv" style="box-sizing: border-box; color: teal;">$_TMP_PWD
</span> <span class="nb" style="box-sizing: border-box; color: #0086b3;">echo</span> <span class="s2" style="box-sizing: border-box; color: #dd1144;">"\nAlpha changes have been reverted with the provided backup file."
</span><span style="font-weight: bold;">}</span></div>
<div class="line" id="LC120" style="box-sizing: border-box; color: #333333; padding-left: 10px;">
</div>
</pre>
<div style="text-align: left;">
~</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The full [WIP] shell-profile for golang utilities is at:</div>
<a href="https://github.com/abhishekkr/tux-svc-mux/blob/master/shell_profile/a.golang.sh" target="_blank">https://github.com/abhishekkr/tux-svc-mux/blob/master/shell_profile/a.golang.sh</a></div>
</div>
abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com1tag:blogger.com,1999:blog-7154053114439400556.post-17079505743720085402013-12-05T07:49:00.000-08:002013-12-05T07:49:02.939-08:00go get pkg ~ easy made easier for project dependency management<div dir="ltr" style="text-align: left;" trbidi="on">
For past sometime I've been trying out ways to improve practices upon awesome capabilities from GoLang. One of the things have been having a 'bundle install' (for ruby folks) or 'pip require -e' (for python folks) style capability... something that just refers to an text file part of source code and plainly fetches all the dependencies path mentioned in there (for all others).<br />
<br />
It and some other bits can be referred here...<br />
<a href="https://github.com/abhishekkr/tux-svc-mux/blob/master/shell_profile/a.golang.sh#L31">https://github.com/abhishekkr/tux-svc-mux/blob/master/shell_profile/a.golang.sh#L31</a><br />
<br />
It's a shell (bash) function that can be added to your Shell/System Profile files and used...<br />
<br />
<pre><div class="line" id="LC31" style="background-color: #ffffcc; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; line-height: 18px; padding-left: 10px;">
<span style="color: #333333;">go_get_pkg</span><span class="o" style="color: #333333;"><b>(){
</b></span><span style="background-color: transparent; color: #333333;"> </span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;">if</span><span style="background-color: transparent; color: #333333;"> </span><span class="o" style="background-color: transparent; color: #333333; font-weight: bold;">[</span><span style="background-color: transparent; color: #333333;"> </span><span class="nv" style="background-color: transparent; color: teal;">$# </span><span style="background-color: transparent; color: #333333;">-eq 0 </span><span class="o" style="background-color: transparent; color: #333333; font-weight: bold;">]</span><span class="p" style="background-color: transparent; color: #333333;">;</span><span style="background-color: transparent; color: #333333;"> </span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;">then
</span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;"> if</span><span style="background-color: transparent; color: #333333;"> </span><span class="o" style="background-color: transparent; color: #333333; font-weight: bold;">[</span><span style="background-color: transparent; color: #333333;"> -f </span><span class="s2" style="background-color: transparent; color: #dd1144;">"$PWD/go-get-pkg.txt"</span><span style="background-color: transparent; color: #333333;"> </span><span class="o" style="background-color: transparent; color: #333333; font-weight: bold;">]</span><span class="p" style="background-color: transparent; color: #333333;">;</span><span style="background-color: transparent; color: #333333;"> </span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;">then
</span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;"> </span><span class="nv" style="background-color: transparent; color: teal;">PKG_LISTS</span><span class="o" style="background-color: transparent; color: #333333; font-weight: bold;">=</span><span class="s2" style="background-color: transparent;"><span style="color: #dd1144;">"$PWD/go-get-pkg.txt"
</span></span><span style="background-color: transparent; color: #333333;"> </span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;">else
</span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;"> </span><span style="background-color: transparent; color: #333333;">touch </span><span class="s2" style="background-color: transparent;"><span style="color: #dd1144;">"$PWD/go-get-pkg.txt"
</span></span><span style="background-color: transparent; color: #333333;"> </span><span class="nb" style="background-color: transparent; color: #0086b3;">echo</span><span style="background-color: transparent; color: #333333;"> </span><span class="s2" style="background-color: transparent;"><span style="color: #dd1144;">"Created GoLang Package empty list $PWD/go-get-pkg.txt"
</span></span><span style="background-color: transparent; color: #333333;"> </span><span class="nb" style="background-color: transparent; color: #0086b3;">echo</span><span style="background-color: transparent; color: #333333;"> </span><span class="s2" style="background-color: transparent; color: #dd1144;">"Start adding package paths as separate lines."</span><span style="background-color: transparent; color: #333333;"> </span><span class="o" style="background-color: transparent; color: #333333; font-weight: bold;">&&</span><span style="background-color: transparent; color: #333333;"> </span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;">return </span><span style="background-color: transparent; color: #333333;">0
</span><span style="background-color: transparent; color: #333333;"> </span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;">fi
</span><span style="background-color: transparent; color: #333333; font-weight: bold;"> else
</span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;"> </span><span class="nv" style="background-color: transparent; color: teal;">PKG_LISTS</span><span class="o" style="background-color: transparent; color: #333333; font-weight: bold;">=(</span><span class="nv" style="background-color: transparent; color: teal;">$@</span><span class="o" style="background-color: transparent; color: #333333;"><b>)
</b></span><span style="background-color: transparent; color: #333333;"> </span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;">fi
</span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;"> for </span><span style="background-color: transparent; color: #333333;">pkg_list in </span><span class="nv" style="background-color: transparent; color: teal;">$PKG_LISTS</span><span class="p" style="background-color: transparent; color: #333333;">;</span><span style="background-color: transparent; color: #333333;"> </span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;">do
</span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;"> </span><span style="background-color: transparent; color: #333333;">cat </span><span class="nv" style="background-color: transparent; color: teal;">$pkg_list</span><span style="background-color: transparent; color: #333333;"> </span><span class="p" style="background-color: transparent; color: #333333;">|</span><span style="background-color: transparent; color: #333333;"> </span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;">while </span><span class="nb" style="background-color: transparent; color: #0086b3;">read </span><span style="background-color: transparent; color: #333333;">pkg_path</span><span class="p" style="background-color: transparent; color: #333333;">;</span><span style="background-color: transparent; color: #333333;"> </span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;">do
</span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;"> </span><span class="nb" style="background-color: transparent; color: #0086b3;">echo</span><span style="background-color: transparent; color: #333333;"> </span><span class="s2" style="background-color: transparent; color: #dd1144;">"fetching golag package: go get ${pkg_path}"</span><span class="p" style="background-color: transparent; color: #333333;">;
</span><span style="background-color: transparent; color: #333333;"> </span><span class="nb" style="background-color: transparent; color: #0086b3;">echo</span><span style="background-color: transparent; color: #333333;"> </span><span class="nv" style="background-color: transparent; color: teal;">$pkg_path</span><span style="background-color: transparent; color: #333333;"> </span><span class="p" style="background-color: transparent; color: #333333;">|</span><span style="background-color: transparent; color: #333333;"> xargs go get
</span><span style="background-color: transparent; color: #333333;"> </span><span class="k" style="background-color: transparent; color: #333333; font-weight: bold;">done
</span><span style="background-color: transparent; color: #333333; font-weight: bold;"> done
</span><span style="background-color: transparent; color: #333333; font-weight: bold;">}</span></div>
<div class="line" id="LC49" style="color: #333333; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; line-height: 18px; padding-left: 10px;">
<span style="color: black; font-family: 'Times New Roman'; font-size: small; line-height: normal; white-space: normal;">---</span></div>
<div class="line" id="LC49" style="padding-left: 10px;">
<span class="o"><div style="color: black; font-family: 'Times New Roman'; font-weight: normal; line-height: normal; white-space: normal;">
<span style="font-size: large;">
</span></div>
<div style="color: black; font-family: 'Times New Roman'; line-height: normal; white-space: normal;">
<span style="font-size: large;"><b>What it do?</b></span></div>
<span style="font-family: Trebuchet MS, sans-serif;">If ran without any parameters.
It checks for current working directory for a file called 'go-get-pkg.txt'.
If not found creates one empty file by that name. To be done at initialization of project.
If found, then it iterates through each line and pass it directly to "get get ${line}".
If ran with parameters. Each parameter is treated as path to files similar to 'go-get-pkg.txt' and similar action as explained previously is performed on each file.</span></span></div>
<div class="line" id="LC49" style="padding-left: 10px;">
<span class="o"><span style="font-family: Arial, Helvetica, sans-serif;">
</span>
<div style="color: black; font-family: 'Times New Roman'; font-size: medium; font-weight: normal; line-height: normal; white-space: normal;">
Sample 'go-get-pkg.txt' file</div>
<div style="color: black; font-family: 'Times New Roman'; font-size: medium; font-weight: normal; line-height: normal; white-space: normal;">
source: <a href="https://github.com/abhishekkr/goshare/blob/master/go-get-pkg.txt" style="font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-weight: bold; line-height: 18px;">https://github.com/abhishekkr/goshare/blob/master/go-get-pkg.txt</a></div>
<pre style="color: #333333; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-weight: normal; line-height: 18px;"><div class="line" id="LC1" style="padding-left: 10px;">
-tags zmq_3_x github.com/alecthomas/gozmq
github.com/abhishekkr/levigoNS
github.com/abhishekkr/goshare</div>
</pre>
<div style="color: black; font-family: 'Times New Roman'; font-size: medium; font-weight: normal; line-height: normal; white-space: normal;">
</div>
<div style="color: black; font-family: 'Times New Roman'; font-size: medium; font-weight: normal; line-height: normal; white-space: normal;">
---</div>
</span></div>
</pre>
</div>
abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com1tag:blogger.com,1999:blog-7154053114439400556.post-83077904777447669102013-11-15T06:21:00.000-08:002013-11-17T05:58:43.772-08:00systemd enabled lightweight NameSpace Containers ~ QuickStart Guide<div dir="ltr" style="text-align: left;" trbidi="on">
<b><a href="http://www.freedesktop.org/wiki/Software/systemd/" target="_blank">systemd</a></b> (<i>for some time now</i>) provides a powerful <a href="http://en.wikipedia.org/wiki/Chroot" target="_blank">chroot</a> alternative to <a href="http://www.linux.com/" target="_blank"><b>linux</b></a> users for <span style="font-size: large;">creating quick and lightweight system containers</span> using power of <a href="https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt" target="_blank"><b>cgroups</b></a> and <a href="http://0pointer.de/blog/projects/socket-activated-containers.html" target="_blank"><b>socket activation</b></a>.<br />
<br />
There is a lot more to "<b><i>systemd</i></b>" than this, but that's for some other post. Until then can explore it, <i><a href="http://www.freedesktop.org/wiki/Software/systemd/" target="_blank">starting here</a></i>.<br />
<br />
There is a utility "systemd-nspawn" provided by systemd which acts as container manager. This is what can be used to easily spawn a new linux container and manage it. It has been updated with (the systemd's amazing trademark feature) Socket Activation.<br />
<br />
This enables any container to make parent/host's systemd instance to listen at different service ports for itself. Only when those service ports receive a connection, these container will spawn and act to it. Voila, resource utilization and scalability concepts.<br />
More of this can read in detail at: <a href="http://0pointer.de/blog/projects/socket-activated-containers.html">http://0pointer.de/blog/projects/socket-activated-containers.html</a><br />
<br />
Here we'll see some way to quickly start using it via some custom made commands.<br />
All the script commands used here can referred from <a href="https://github.com/abhishekkr/tux-svc-mux/blob/master/shell_profile/a.virt.sh">https://github.com/abhishekkr/tux-svc-mux/blob/master/shell_profile/a.virt.sh</a> as well.<br />
<br />
Just download and source the linked script in your shell, and the commands told here will be available...<br />
And yes, your system also need to be running <b>systemd</b> already.<br />
<br />
Currently this just lets you create <b><a href="https://www.archlinux.org/" target="_blank">archlinux</a></b> containers, will soon create different containers and make the script mature.<br />
<br />
In case you don't have any created container already, or wanna create a new one...<br />
<blockquote class="tr_bq">
<span style="background-color: white; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; line-height: 16px; white-space: pre;">$ <b>nspawn-arch</b></span></blockquote>
To list names of all created containers...<br />
<blockquote class="tr_bq">
<span style="background-color: white; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; line-height: 16px; white-space: pre;">$ <b>nspawn-ls</b></span></blockquote>
<div>
To stop a running container...</div>
<div>
<blockquote class="tr_bq">
<span style="background-color: white; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; line-height: 16px; white-space: pre;">$ <b>nspawn-stop <name as="" by="" conatiner="" listed="" nspawn-ls="" of=""></name></b></span></blockquote>
<div>
To start an already created conatiner</div>
<div>
<blockquote class="tr_bq">
<span style="background-color: white; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; line-height: 16px; white-space: pre;">$ <b>nspawn-start </b></span><b style="font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; line-height: 16px; white-space: pre;"><name as="" by="" conatiner="" listed="" nspawn-ls="" of=""></name></b></blockquote>
<div>
<br /></div>
---<br />
<br />
<div>
<script src="https://gist.github.com/abhishekkr/7484912.js"></script>
</div>
<br />
---<br />
<br /></div>
</div>
</div>
abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com2tag:blogger.com,1999:blog-7154053114439400556.post-62400262557055099662013-07-26T17:50:00.002-07:002013-08-01T11:29:59.744-07:00Puppet ~ a beginners concept guide (Part 4) ~ Where is my Data<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: right;">
<span style="background-color: white; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"><i>parts of the "Puppet ~ Beginner's Concept Guide" to read before ~ <br /><a href="http://justfewtuts.blogspot.in/2012/05/puppet-beginners-concept-guide-part-1.html">Part#1</a>: intro to puppet, <a href="http://justfewtuts.blogspot.in/2012/07/puppet-beginners-concept-guide-part-2.html">Part#2</a>: intro to modules, and <a href="http://justfewtuts.blogspot.in/2012/08/puppet-beginners-concept-guide-part-3.html">Part#3</a>: modules much more.</i></span></div>
<br />
<span class="Apple-style-span" style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;">Puppet</span><br />
<span class="Apple-style-span" style="background-color: #fff9ee; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 15px; line-height: 21px;"><b><span class="Apple-style-span" style="font-size: xx-small;">beginners concept guide</span></b></span><span style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"> (Part <b>4</b>)</span><br />
<b style="color: #222222; font-family: Times, 'Times New Roman', serif; font-size: 15px; line-height: 21px;"><i><br /></i></b>
<br />
<h2 style="text-align: left;">
<b style="color: #222222; font-family: Times, 'Times New Roman', serif; font-size: 15px; line-height: 21px;"><i>Where is my Data?</i></b></h2>
<span style="background-color: white; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"></span>When I started my Puppet-ry, the examples I used to see had all configuration data buried inside the DSL code of manifests, people were trying to use inheritance to push down data. Then got to see a design pattern in puppet manifests keeping out separate parameters manifest for configuration variables. Then came along the External Data lookup via CSV files as a Puppet function. Then with enhancements in puppet and other modules came along more.<br />
<br />
<span style="background-color: white; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">Below are few usable to fine ways utilizing separate data sources within your manifests,</span><br />
<span style="background-color: white; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"><br /></span>
<br />
<span style="background-color: white; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">Here, we will see usage styles of data for Puppet Manifests, Extlookup CSV, Hiera, Plug-in Facts and PuppetDB.</span><br />
<h3 style="text-align: left;">
<a href="http://docs.puppetlabs.com/guides/parameterized_classes.html#appendix-smart-parameter-defaults" style="font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; line-height: 21px;" target="_blank"><span style="font-size: 25px;">params-manifest</span></a><span style="background-color: white; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">:</span></h3>
<br />
It is the very basic way of separating out data from your functionality code, and the preferred way for in-future growing value-set type of data. It will keep it separate from the code since start. Once the requirement is at a level to have varied value to inferred based on environment/domain/fqdn/operatingsystem/[any-facter], it can be extracted to any preferred ways given below and just looked-up here. That would avoid changing the main (sub)module-code.<br />
<span style="background-color: #f0f0f0; font-family: helvetica, arial, freesans, clean, sans-serif; font-size: 13px; line-height: 18.66666603088379px;">[ Gist-Set Externalize into Params Manifest: </span><a href="https://gist.github.com/3683955">https://gist.github.com/3683955</a> ]<br />
Say you are providing httpd::git sub-module for httpd module placing a template generated config file using params placed data...<br />
<div>
```<br />
<br />
File: httpd/manifests/git.pp<br />
it includes the params submodule to access the data<br />
<script src="https://gist.github.com/3683955.js?file=git.pp"></script><br />
File: httpd/templates/mynode.conf.erb<br />
<script src="https://gist.github.com/3683955.js?file=mynode.conf.erb"></script><br />
File: httpd/manifests/params.pp<br />
it actually is just another submodule to only handle data<br />
<script src="https://gist.github.com/3683955.js?file=params.pp"></script><br />
Use it: run_puppet.sh<br />
<script src="https://gist.github.com/3683955.js?file=run_puppet.sh"></script>
</div>
<br />
```<br />
<span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;"><span style="font-size: 15px; line-height: 21px;">_</span></span><br />
<h3>
<a href="http://docs.puppetlabs.com/references/2.6.8/function.html#extlookup" style="font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; line-height: 21px;" target="_blank"><span style="font-size: 25px;">extlookup-csv</span></a><span style="background-color: white; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">:</span></h3>
<br />
If you think your data would suit to a (key,value) CSV format being extracted to data files.Puppet need to be told the location for CSV files need to be looked up for key, and fetch the value assigned to it in that file.<br />
Names given to these CSV files would matter to Puppet while looking up the values from all present CSV files. Puppet need to be given hierarchy order for these file-names to look for the key and the order could involve variable names.<br />
<br />
For E.g. say you have a CSV by name of HOSTNAME, ENVIRONMENT and a common file, with hierarchy specified in respective order too. Then Puppet will first look for the queried Key in CSV by HOSTNAME, if not found looks up in ENVIRONMENT named file and after not finding it there goes looking into common file. If it doesn't find the key in any of those files, it returns the default value if specified in the 'extlookup(key, default_value)' method like this. If there is no default value also, Puppet will raise an exception for no value to return.<br />
<br />
[ Gist-Set Externalize into Hierarchical CSV Files: <a href="https://gist.github.com/3684010">https://gist.github.com/3684010</a> ]<br />
<br />
It's the same example as for params with a flavor of ExtData. Here you'll notice a<b> 'common.csv' </b>external data file providing a default set of values. Then there is also a <b>'env_testnode.csv' </b>file overriding the only required changed value. Now as in<b> 'site.pp'</b> file, precedence of <b>'env_%{environment}'</b> file is higher than<b> 'common'</b>, the <b>'httpd::git' </b>would look-up all values first from <b>'env_testnode.csv'</b> and if not found there would goto<b> 'common.csv'</b>. Hence would end-up overriding <b>'httpd_git_url'</b> value from<b> 'env_testnode.csv'</b>. <br />
<div>
```<br />
<script src="https://gist.github.com/abhishekkr/3684010.js"> </script>
```</div>
<br />
<b>extlookup()</b> method used here is available as a <b>Puppet Parser Function</b>, you would read more in <b>Part#5 Custom Puppet Plug-Ins</b> on how to create your own functions<br />
_<br />
<br />
<div style="text-align: left;">
<a href="http://projects.puppetlabs.com/projects/hiera/" style="font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;" target="_blank"><span style="font-size: 25px;"><b>hiera-and-backends</b></span></a><span style="background-color: white; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">:</span><span style="background-color: white; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; line-height: 21px;"><br /></span></div>
<div style="text-align: left;">
</div>
<div style="text-align: left;">
<span style="font-family: inherit;"><span style="font-weight: normal;"><br /></span><span style="font-weight: normal;">Hiera is a pluggable hierarchical data storage for Puppet. It was started to provide a better external data storage support than Ext-lookup feature with data formats other than CSV too.</span></span><br />
This brings in the essence of ENC for data retrieval without having to write one.<br />
<br />
Data look-up happens in a hierarchy provided by configuration with self scope resolution mechanism.<br />
<span style="font-family: inherit;"><span style="font-weight: normal;"><br /></span></span>
<span style="font-family: inherit;"><span style="font-weight: normal;">It enables Puppet to fetch data from varied external data sources using it's different backends (like local files, redis, http protocol) which can be added on to if needed.</span></span><br />
The 'http' backend in turn enables support for data store from any service (couchdb, <b><i><a href="http://beingasysadmin.wordpress.com/2012/11/24/hiera-http-and-riak/" target="_blank">riak</a></i></b>, web-app or so) to provide data.</div>
<div style="text-align: left;">
<span style="font-family: inherit;"><span style="font-weight: normal;"><br /></span></span></div>
<div style="text-align: left;">
File "<a href="https://gist.github.com/abhishekkr/6133012#file-conf_dir-hiera-yaml" target="_blank">hiera.yaml</a>" from Gist below is an example of hiera configuration to be placed in puppet's configuration directory. The highlights of this configuration are ":backends:", backend source and ":hierarchy:". Multiple backend can be used at same time, their order of listing mark their order of look-up. Hierarchy configures the order for data look-up by scope.<br />
<br />
Then depending on what backend you have added, you need to add their source/config to look-up data at.<br />
Here we can see configuration for using local "yaml", "json" files. Look-up data from Redis server (<i><a href="https://gist.github.com/abhishekkr/6133012#file-prepare_redis-sh" target="_blank">it will set-up datasets for redis usage for current example</a></i>) with authentication in place. And looking up data from any "<b><a href="https://github.com/crayfishx/hiera-http" target="_blank">http</a></b>" service with hierarchy as the ":paths:" value.<br />
You can even use <a href="https://github.com/crayfishx/hiera-gpg" target="_blank">GPG protected data as backend</a>, but that is a bit messy to use.<br />
<br />
Placing ".yaml" and ".json" from Gist at intended provider location.<br />
The running "<b><i><a href="https://gist.github.com/abhishekkr/6133012#file-use_hiera-sh" target="_blank">use_hiera.sh</a></i></b>" would make you show the magic from this example on hiera.</div>
<div style="text-align: left;">
<span style="font-family: inherit;"><span style="font-weight: normal;"><br /></span></span>
<span style="font-family: inherit;"><span style="font-weight: normal;">```Gist</span></span><br />
<div>
<script src="https://gist.github.com/abhishekkr/6133012.js"></script>
</div>
<span style="font-family: inherit;"><span style="font-weight: normal;">```</span></span><br />
[Gist-Set Using Hiera with Masterless Puppet set-up: <a href="https://gist.github.com/abhishekkr/6133012">https://gist.github.com/abhishekkr/6133012</a> ]<br />
<span style="font-family: inherit;"><span style="background-color: white; color: #222222; line-height: 21px;">_</span></span></div>
<div style="text-align: left;">
<span style="font-family: inherit;"><span style="background-color: white; color: #222222; line-height: 21px;"><br /></span></span></div>
<h3 style="text-align: left;">
<a href="http://docs.puppetlabs.com/guides/custom_facts.html" style="font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;" target="_blank"><span style="font-size: 25px;">plugin-facts</span></a><span style="background-color: white; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">:</span></h3>
<div style="text-align: left;">
<span style="font-family: inherit;"><span style="font-weight: normal;"><br /></span><span style="font-weight: normal;">Every system has its own set of information facter (</span></span><a href="http://projects.puppetlabs.com/projects/facter">http://projects.puppetlabs.com/projects/facter</a>)<span style="font-family: inherit;"> by default made available to puppet. Puppet also enable DevOps people to set custom facter to be used in modules.<br />The power of these computed Facters is they can use full ruby-power to use local/remote plain/encrypted data over REST/Database/API/anyway available channel.</span></div>
<div style="text-align: left;">
<span style="font-family: inherit;">These require the power of Puppet Custom Plug-Ins (</span><a href="http://docs.puppetlabs.com/guides/custom_facts.html">http://docs.puppetlabs.com/guides/custom_facts.html</a>). The ruby file doing this would go at 'MODULE/lib/puppet/facter' and would get loaded by the 'pluginsync=true' in action.</div>
<div style="text-align: left;">
<span style="font-family: inherit;"><span style="font-weight: normal;">Way to set a Facter in such Ruby code is just...</span></span></div>
<blockquote class="tr_bq">
my_value = 'all ruby code to compure it'<br />
Facter.add(mykey) do<br />
setcode do<br />
my_value<br />
end<br />
end</blockquote>
<div>
<div style="text-align: left;">
<span style="font-family: inherit;"><span style="font-weight: normal;">.....all the rest of code there need to compute the value to be set, or even key-set.</span></span></div>
<div style="text-align: left;">
<span style="font-family: inherit;"><span style="font-weight: normal;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-family: inherit;"><span style="font-weight: normal;">[Gist-Set Externalize Data receiving as Facter: </span></span><a href="https://gist.github.com/3684968">https://gist.github.com/3684968</a><span style="font-family: inherit;"> ]</span></div>
<div style="text-align: left;">
<span style="font-family: inherit;"><span style="font-weight: normal;"><br /></span><span style="font-weight: normal;">Same 'httpd::git' example revamped to use Custom Facter as </span><span style="background-color: white; color: #222222; line-height: 21px;"><br /></span></span></div>
<div>
```<br />
<script src="https://gist.github.com/abhishekkr/3684968.js"></script>
```</div>
<div style="text-align: left;">
<span style="background-color: white; color: #222222; line-height: 21px;"><span style="font-family: inherit;">There is also another way to provide a Facter in Puppet Catalog, that can be done by providing an Environment variable with capitalized Facter name pre-fixed by 'FACTER_' and the value which it's supposed to have.</span><br /><span style="font-family: inherit;">For E.g. </span><b style="font-family: inherit;"># </b><span style="font-family: Courier New, Courier, monospace;">FACTER_MYKEY=$my_value puppet apply --modulepath=$MODULEPATH -e "include httpd::git"</span><br /><span style="font-family: inherit;">_</span></span></div>
<div style="text-align: left;">
<span style="font-family: inherit;"><span style="background-color: white; color: #222222; line-height: 21px;"><br /></span></span></div>
<h3 style="text-align: left;">
<a href="http://puppetlabs.com/blog/introducing-puppetdb-put-your-data-to-work/" style="font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;" target="_blank"><span style="font-size: 25px;">puppetdb</span></a><span style="background-color: white; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">:</span></h3>
<div style="text-align: left;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: left;">
It's a beautiful addition to Puppet component set. Something that have been missing for long and possibly the thing because of which I delayed this post by half year.<br />
It enables the 'storeconfig' power without the Master, provides a support of trusted DB for infrastructure-related data needs and thus best suited of all.<br />
<br />
To set-up 'puppetdb' on a node follow the <a href="http://docs.puppetlabs.com/puppetdb/latest/index.html#install-it-now" target="_blank">PuppetLabs has a nice documentation</a>.</div>
<div style="text-align: left;">
<span style="font-family: inherit;"><span style="font-weight: normal;">To set-up a decent example for master-less puppet mode, follow the given steps</span></span><br />
<span style="font-family: inherit;"><span style="font-weight: normal;"><br /></span></span>
<span style="font-family: inherit;"><span style="font-weight: normal;">Place the 2 '.conf' and 1 '.yaml' file in Puppet's configuration directory.</span></span><br />
<span style="font-family: inherit;"><span style="font-weight: normal;">The </span><b><i><a href="https://gist.github.com/abhishekkr/6114760#file-prepare_puppetdb-sh" target="_blank">shell script</a></i></b><span style="font-weight: normal;"> would prepare the node with PuppetDB service for masterless puppet usage scenario.</span></span><br />
<span style="font-family: inherit;"><span style="font-weight: normal;"><br /></span></span>
<span style="font-family: inherit;"><span style="font-weight: normal;"><a href="https://gist.github.com/abhishekkr/6114760#file-conf_dir-puppet-conf" target="_blank">Puppet config</a> setting storeconfig to 'puppetdb' enables saving of exported resources to it. The 'reports' config their would push the puppet apply reports to the database.</span></span><br />
<span style="font-family: inherit;"><a href="https://gist.github.com/abhishekkr/6114760#file-conf_dir-puppetdb-conf" target="_blank">PuppetDB config</a><span style="font-weight: normal;"> makes Puppet aware of the host and port to connect database at.</span></span><br />
<span style="font-family: inherit;"><span style="font-weight: normal;">The facts setting on <a href="https://gist.github.com/abhishekkr/6114760#file-conf_dir-routes-yaml" target="_blank">routes.yaml</a> enable PuppetDB to be used in a masterless mode.</span></span><br />
<span style="font-family: inherit;"><span style="font-weight: normal;"><br /></span></span>
```<br />
<div>
<script src="https://gist.github.com/abhishekkr/6114760.js"></script>
</div>
```<br />
<span style="font-family: inherit;"><span style="font-weight: normal;"><br /></span></span>
<span style="font-family: inherit;"><span style="font-weight: normal;">[Gist-Set Using PuppetDB with Masterless Puppet set-up: </span></span><a href="https://gist.github.com/abhishekkr/6114760">https://gist.github.com/abhishekkr/6114760</a> ]<br />
<span style="font-family: inherit;"><span style="font-weight: normal;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-family: inherit;"><span style="font-weight: normal;">Now running anything say like...</span></span><br />
<span style="font-family: inherit;"><b><i>puppet apply -e 'package{"vim": }'</i></b></span><br />
and beautiful to that '<b><i>export resources</i></b>' would work like a charm using PuppetDB.<br />
The puppet.conf accompanied will make reports dumped to PuppetDB as well.<br />
<br /></div>
<div style="text-align: left;">
<span style="font-family: inherit;"><span style="background-color: white; color: #222222; line-height: 21px;">_</span></span></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<br /></div>
<span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;"><span style="font-size: 15px; line-height: 21px;">There's a <a href="http://puppetlabs.com/blog/the-problem-with-separating-data-from-puppet-code/" target="_blank">fine article</a> on the same by <a href="http://puppetlabs.com/blog/the-problem-with-separating-data-from-puppet-code/" target="_blank">PuppetLabs</a>...</span></span></div>
</div>
abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com6tag:blogger.com,1999:blog-7154053114439400556.post-71126998105965876342013-05-31T15:32:00.002-07:002013-05-31T15:59:36.016-07:00Testing Chaos with Automated Configuration Management solutions<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div style="text-align: left;">
<span style="font-family: inherit;">No noise making.</span></div>
<div style="text-align: left;">
<span style="font-family: inherit;"><br />But let's be real, think of the count of community contributed (or mysterious closed-and-sold 3rd Party) services, frameworks, library and modules put to use for managing your ultra-cool self-healing self-reliant scalable Infrastructure requirements. Now with so many cogs collaborating in the infra-machine, a check on their collaboration seems rather mandatory like any other integration test for your in-house managed service. </span><span style="font-family: inherit;">After all that was key idea behind having automated configuration management itself.</span></div>
<div style="text-align: left;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: left;">
<span style="font-family: inherit;">Now the utilities like Puppet/Chef have been out there accepted and used by dev & ops folks for quite some time now.</span></div>
<div style="text-align: left;">
<span style="font-family: inherit;">But the issue with the initially seen amateur testing styles is it evolved from the non-matching frame of 'Product' oriented unit/integration/performance testing. 'Product' oriented testing focus more on what happens inside the coded logic and less on how user gets affected by product.<br />Most of the initial tools released for testing logic developed in Chef/Puppet were RSpec/Cucumber inspired Product testing pieces. Now for the major part of installing a package, restarting a service or pushing artifacts these tests are almost non-required as the main functionality for per-say installing package_abc is already tested inside the framework being used.<br />So coding to "ask" to install package_abc and testing if it has been asked seems futile.<br /><span style="color: #222222;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-family: inherit;"><span style="color: #222222;">That's the shift. The logic developed for Infrastructure acts as a </span><span style="color: #222222;">glue to all other applications created in house and 3rd party. </span><span style="color: #222222;">Here in Infrastructure feature development there is more to test for the effect it has on the it's users (software/hardware) and less on internal changes (dependencies and dynamic content). Now the stuff in parentheses here means a lot more than seems... let's get into detail of it.</span></span></div>
<div style="background-color: white; text-align: left;">
<br /></div>
<h3 style="text-align: left;">
<i>Real usability of Testing is based on keeping sanctity of WHAT needs to be tested WHERE.</i></h3>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif;">
<br /></div>
<div style="background-color: white; color: #222222;">
<span style="font-family: inherit;">Software/Hardware services that collaborate with the help of Automated Infrastructure logic needs major focus of testing. These services can be varying from the</span></div>
<div style="background-color: white; color: #222222;">
<ul style="text-align: left;">
<li><span style="font-family: inherit;">in-house 'Product', that is the central component you are developing</span></li>
<li><span style="font-family: inherit;">3rd Party services it collaborates with,</span></li>
<li><span style="font-family: inherit;">external services it utilizes for what it doesn't host,</span></li>
<li><span style="font-family: inherit;">operating system that it supports and Ops-knows what not.</span></li>
</ul>
</div>
<div style="background-color: white; color: #222222;">
<span style="font-family: inherit;"><br /></span></div>
<div style="background-color: white; color: #222222;">
<span style="font-family: inherit;">Internal changes mainly revolve around</span></div>
<div style="background-color: white; color: #222222;">
<ul style="text-align: left;">
<li><span style="font-family: inherit;">Resources/Dependencies getting called in right order and grouped for specific state.</span></li>
<li><span style="font-family: inherit;">It also relates to correct generation/purging of dynamic content, that content can itself range as</span></li>
<ul>
<li><span style="font-family: inherit;">non-corrupt configuration files generated of a template</span></li>
<li><span style="font-family: inherit;">format of sent configuration data from one Infra-component to another for reflected changes</span></li>
<li><span style="font-family: inherit;">dynamically creating/destroying service instances in case of auto-scalable infrastructure</span></li>
</ul>
</ul>
</div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif;">
<i><br /></i></div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif;">
<i><br /></i></div>
<h3 style="text-align: left;">
<i>One can decide <b>HOW,</b> on ease and efficiency basis.</i></h3>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif;">
<b><br /></b></div>
<div style="background-color: white; color: #222222;">
<span style="font-family: inherit;"><b>Unit Tests</b> work for the major portion of 'Internal Changes' mentioned before using <a href="https://github.com/acrmp/chefspec" target="_blank">chefspec</a>, <a href="http://rubygems.org/gems/rspec-chef" target="_blank">rspec-chef</a>, <a href="http://rubygems.org/gems/rspec-puppet" target="_blank">rspec-puppet</a> like libraries are good enough. They can very well test the dependency order and grouping management as well as the different data effect on non-corrupt configuration generation from templates.</span></div>
<div style="background-color: white; color: #222222;">
<span style="font-family: inherit;"><br /></span></div>
<div style="background-color: white; color: #222222;">
<span style="font-family: inherit;"><br /></span></div>
<div style="background-color: white; color: #222222;">
<span style="font-family: inherit;"><b>Integration Tests</b> in this perspective are a of a bit interesting and evolutionary nature. Here we have to ensure the "glue" functionality we talked about for Software/Hardware service is working properly. These will confirm that every type of required machine role/state can be achieved flawlessly, call them <i><b>'</b></i><i><b>State Generation Test'</b></i>. They also need to confirm the <i><b>'Reflected Changes Test'</b></i> across Infra-component as mentioned in Internal changes.</span></div>
<div style="background-color: white; color: #222222;">
<span style="font-family: inherit;">Now utilities like <a href="https://github.com/opscode/test-kitchen" target="_blank">test-kitchen</a>/<a href="https://github.com/portertech/kitchen-docker" target="_blank">docker</a> in collaboration with <a href="http://www.vagrantup.com/" target="_blank">vagrant</a>, <a href="http://www.docker.io/gettingstarted/" target="_blank">docker</a>, etc. help placing them in your <a href="http://en.wikipedia.org/wiki/Continuous_integration" target="_blank">Continuous Integration</a> pipeline. This would even help in testing same service across multiple linux distros if that's the plan to support.</span></div>
<div style="background-color: white; color: #222222;">
<span style="font-family: inherit;">Library '<a href="http://serverspec.org/" target="_blank">ServerSpec</a>' is also a little nifty piece to write quick final state check scripts.</span></div>
<div style="background-color: white; color: #222222;">
<span style="font-family: inherit;">Then final set of Integration Testing is implemented in form of <i><b>Monitoring</b></i> on your all managed/affecting Infrastructure components. This is the final and ever-running Integration Test.</span></div>
<div style="background-color: white; color: #222222;">
<span style="font-family: inherit;"><br /></span></div>
<div style="background-color: white; color: #222222;">
<span style="font-family: inherit;"><br /></span></div>
<div style="background-color: white;">
<span style="font-family: inherit;"><b style="color: #222222;">Performance Tests</b><span style="color: #222222;">, yes even they are required for it. Tools like </span><a href="https://github.com/Netflix/SimianArmy" style="color: #222222;" target="_blank">ChaosMonkey</a><span style="color: #222222;"> enable you to enable your Infra to be self-healing and auto-scalable. Should be load-test noticing dynamic containers count and behavior if auto-scalability is a desired functionality too</span><span style="color: #222222;">.</span></span></div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">
<br /></div>
</div>
abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com8tag:blogger.com,1999:blog-7154053114439400556.post-5899264826485287912013-04-24T11:49:00.000-07:002013-04-24T11:49:07.249-07:00Beginner's Guide to OpenStack : Basics of Nova [Part 2]<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: right;">
<i>parts of <b>Beginner's Guide to OpenStack</b> to read before this ~</i></div>
<div style="text-align: right;">
<a href="http://justfewtuts.blogspot.com/2013/04/beginners-guide-to-openstack-basics.html" target="_blank"><i>Part1: Basics</i></a></div>
<br />
[Part.2 Basics of Nova] Beginner's Guide to OpenStack<br />
<br />
<b># Nova?</b><br />
It's the main fabric controller for IaaS providing Cloud Computing Service by OpenStack. Took its first baby steps in NASA. Contributed to OpenSource and became most important component of OpenStack.<br />It built of multiple components performing different tasks turning End User's API request into a virtual machine service. All these components run in a non-blocking message based architecture, and can be run off from same or different locations with just access to same message queue service.<br />
<br />
---<br />
<br />
<b># Components?</b><br />
<br class="Apple-interchange-newline" />
Nova stores states of virtual machines in a central database. It's optimal for small deployments. Nova is moving towards multiple data stores with aggregation for high scale requirements.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxwPdkH4RAhqiHkY21Cjcg-o9tRcjtkrehJnG2SJllampCXTDdTO4_gvoc3xrc3kEmpM1Vg9QUpHSNRE1l-ezUNuuz5xG3wBmRKvTOie-9VU8k6jjjCfnpAANb6fumrhHFGLT3Of425VY/s1600/openstack_p2_components.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="115" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxwPdkH4RAhqiHkY21Cjcg-o9tRcjtkrehJnG2SJllampCXTDdTO4_gvoc3xrc3kEmpM1Vg9QUpHSNRE1l-ezUNuuz5xG3wBmRKvTOie-9VU8k6jjjCfnpAANb6fumrhHFGLT3Of425VY/s640/openstack_p2_components.jpg" width="480" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<ul style="text-align: left;">
<li><b>Nova API</b> : supports OpenStack Compute API, Amazon's EC2 API and powerful Admin API (for privileged users). It's used to initiate most of orchestration activities and policies (like Quota). It gets communicated over HTTP, converts the requests to commands further contacting other components via Message Broker and HTTP for ObjectStore. It's a <a href="http://wsgi.readthedocs.org/en/latest/what.html" target="_blank">WSGI</a> application which routes and authenticates requests.</li>
<li><b>Nova Compute</b> : worker daemon taking orders from its Message Broker and perform virtual machine create/delete tasks using Hypervisor's API. It also updates status of its tasks in Database.</li>
<li><b>Nova Scheduler</b> : decides which Nova Compute Host to allot for virtual machine request.</li>
<li><b>Network Manager</b> : worker daemon picking network related tasks from its Message Broker and performing those. <i>OpenStack's Quantum </i>now with <i>Grizzly</i> release can be opted instead of <i>nova-network</i>. Tasks like maintaining IP Forwarding, Network Bridges and VLANs get covered.</li>
<li><b>Volume Manager</b> : handles attach/detach of persistent block storage volumes to virtual machines (similar to Amazon's EBS). This functionality has been extracted to <i>OpenStack's Cinder</i>. It's an ISCSI solution utilizing <a href="http://en.wikipedia.org/wiki/Logical_Volume_Manager_(Linux)" target="_blank">Logical Volume Manager</a>. Network Manager doesn't interfere in Cinder's tasks but need to be setup for Cinder to be used.</li>
<li><b>Authorization Manager</b> : interfaces authorized APIs usage for Users, Projects and Roles. It communicates with <i>OpenStack's KeyStone</i> for details.</li>
<li><b>WebUI</b> : <i>OpenStack's Horizon</i> communicates with Nova API for Dashboard interfacing.</li>
<li><b>Message Broker</b> : All components of Nova communicate with each other in a non-blocking callback-oriented manner using AMQP protocol well supported by RabbitMQ, Apache QPid. There is also emerging support for ZeroMQ integration as Message Queue. It's like central task list shared and updated by all Nova components.</li>
<li><b>ObjectStore</b> : It's a simple file-based storage (like Amazon's S3) for images. This can be replaced with OpenStack's Glance.</li>
<li><b>Database</b> : used to gather build times, run states of virtual machines. It has details around instance types available, networks available (if nova-network), and projects. Any database supported by SQLAlchemy can be used. It's central information hub for all Nova components.</li>
</ul>
<br />
<br />
---<br />
<br />
<b># API Style</b><br />Interface is mostly RESTful. <a href="http://routes.readthedocs.org/en/latest/" target="_blank">Routes (python re-implementation of Rails route system) packages</a> maps URIs to action methods on controller classes.<br />
Each HTTP Request to Compute requires specific authentication credentials required. Multiple authentication schemes can be allowed for a Compute node, provider determines the one to be used.<br />
<br />
---<br />
<br />
<b># Threading Model</b><br />
Uses Green Thread implementation by design using <a href="http://eventlet.net/" target="_blank">eventlet</a> and <a href="http://greenlet.readthedocs.org/en/latest/" target="_blank">greenlet</a> libraries. This results into single process thread for O.S. with it's blocking I/O issues. Though single reduces race conditions to great extent, to eliminate them further in suspicious scenarios use decorator <i>@lockutils.synchronized('lock_name')</i> over methods to be protected from it.<br />
If any action is long-running, it should have methods with desired process-state location triggering eventlet context switch. Placing something like following code-piece will switch context to waiting threads, if any. And will continue on current thread without any delay if there is no other thread in wait.<br />
<blockquote class="tr_bq">
<span style="font-family: inherit;">from eventlet import <a href="http://eventlet.net/doc/modules/greenthread.html?highlight=greenthread#eventlet.greenthread" target="_blank">greenthread</a><br />greenthread.sleep(0)</span></blockquote>
MySQL query uses drivers blocking main process thread. In Diablo release a thread pool was implemented but removed because of trade-off for advantages over bugs.<br />
<br />
---<br />
<br />
<b># Filtering Scheduler</b><br />
In short it's the mechanism used by 'nova-scheduler' to choose the worthy nova-compute host for new required virtual machine to be spawned upon. It prepares a dictionary of unfiltered hosts and weigh their costing for creating required virtual machine(s) request. Then it chooses the least costly host.<br />
Hosts are weighted based on the configuration options for virtual machines.<br />
It's a better practice for customer to ask for large count of required instances together as each request computes weight.<br />
<br />
---<br />
<br />
<b># Message Queue Usage</b><br />
Nova components use <a href="http://en.wikipedia.org/wiki/Remote_procedure_call" target="_blank">RPC</a> to communicate each other via Message Broker using <a href="http://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern" target="_blank">PubSub</a>. Nova implements rpc.call (request/response, API acts as consumer) and rpc.cast (one way, API acts as publisher).<br />
Nova API and Scheduler uses message queue as Invoker, whereas Network and Compute act as workers. Invoker pattern sends messages via rpc.call or rpc.cast. Worker pattern receives messages from queue and respond back to rpc.call with appropriate response.<br />
Nova uses <a href="http://kombu.readthedocs.org/en/latest/introduction.html" target="_blank">Kombu</a> library when interfacing with <a href="http://www.rabbitmq.com/" target="_blank">RabbitMQ</a>.<br />
<br />
---<br />
<br />
<b># Hooks</b><br />
Enable developers to extend Nova capabilities by adding named hooks to Nova code as decorator that will lazily load plug-in code matching hook name (using setuptools entrypoints, it's an extension mechanism). The hook's class definition should have pre and post method.<br />
Don't use hooks when stability is a factor, internal APIs may change.<br />
<br />
---<br />
<br />
<b># Dev Bootstrap</b><br />
To get started with contributing... <a href="https://wiki.openstack.org/wiki/How_To_Contribute#If_you.27re_a_developer.2C_start_here:" target="_blank">read this (OpenStack Wiki on HowToContribute) in detail</a>.<br />
<br />
To get rolling with Nova wheels, system will need to have libvirt and one of the hypervisors (xen/kvm preferred for linux hosts) present.<br />
<blockquote class="tr_bq">
$ git clone git://github.com/openstack/nova.git<br />$ cd nova<br />$ python ./tools/install_venv.py</blockquote>
this will prepare your copy of nova codebase with virtualenv required, now any command you wanna run on this in context of required codebase<br />
<blockquote class="tr_bq">
$ ./tools/with_venv.sh <command></command></blockquote>
<br />
---<br />
<br />
<b># Run My Tests</b><br />
to run the nose tests and pep8 checker, when you are done with virtualenv setup (or that will be initiated first here)... inside 'nova' codebase<br />
<blockquote class="tr_bq">
$ ./run_tests.sh</blockquote>
<br />
---<br />
<br />
<b># Terminology</b><br />
<br />
<ul style="text-align: left;">
<li><i>Server</i>: Virtual Machines created inside Compute System, required Flavor & Image detail.</li>
<li><i>Flavor</i>: Represents unique hardware configurations with disk space, memory and CPU time priority</li>
<li><i>Image</i>: System Image File used to create/rebuild a Server</li>
<li><i>Reboot</i>: Soft Server Reboot sends a graceful shutdown signal. Hard Reboot does power reset.</li>
<li><i>Rebuild</i>: Removes all data on Server and replaces it with specified image. Server's IP Address and ID remains same.</li>
<li><i>Resize</i>: Converts existing server to a different flavor. All resize need to be explicitly confirmed, only then the original server is removed. After 24hrs. delay, there is an automated confirmation.</li>
</ul>
<br />
<br /></div>
abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com2tag:blogger.com,1999:blog-7154053114439400556.post-12855101142003483212013-04-17T16:52:00.000-07:002013-04-18T07:04:17.326-07:00Beginner's Guide to OpenStack : Basics [Part 1]<div dir="ltr" style="text-align: left;" trbidi="on">
# <b>OpenStack?</b><br />
<br />
OpenStack (<a href="http://www.openstack.org/">http://www.openstack.org/</a>) is an OpenSource cloud computing platform that can be used to build up a <b><i><a href="http://www.rackspace.com/cloud/servers/overview_a/" target="_blank">Public</a></i></b> and <a href="http://www.rackspace.com/cloud/private/openstack_software/" target="_blank"><b><i>Private</i></b></a> cloud. As in weaving of various technological components to provide a capability to build a cloud service supporting any use-case and scale.<br />
<br />
Once upon a time RackSpace came into Cloud Services. In some parallel beautiful world, few Pythonistas at NASA started building there own Nova Cloud Compute to handle there own instances. RackSpace bought SliceHost which worked 'somewhat' fine. RackSpace came along with their Swift Object Storage Service and weaved in Nova with few more components around it. More other companies like HP, RedHat, Canonical etc. came along to contribute and benefit from OpenSource cloud.<br />
<br />
It's all Open it can be. Open Source. Open Design. Ope Development. Open Community.<br />
<br />
---<br />
<br />
<b># Quick Hands-On</b><br />
<br />
DevStack (<a href="http://devstack.org/">http://devstack.org/</a>) gives you the easiest fastest way to get all OpenStack components installed, configured and started on any supported O.S. platform.<br />
You can trial-run your app-code in an OpenStack environment at TryStack (<a href="http://trystack.org/">http://trystack.org/</a>).<br />
RedHat RDO (<a href="http://openstack.redhat.com/Main_Page">http://openstack.redhat.com/Main_Page</a>) is also coming in soon making it super easy to get OpenStack running on RHEL-based distros.<br />
---<br />
<br />
<h3 style="text-align: left;">
<b># Components?</b></h3>
<br />
OpenStack Cloud Platform constitutes of mainly following components:<br />
<ul style="text-align: left;">
<li><i><b>Compute</b>:</i> <b>Nova</b><br />Brings up and maintains operations related to virtual server as per requirement.<br />~like aws ec2</li>
<li><i><b>Storage</b>: </i><b>Swift</b><br />Allows you to store, retrieve & remove objects (files).<br />~like aws s3</li>
<li><i><b>Image Registry/Delivery</b>: </i><b>Glance</b><br />Processes metadata for disk images, manages read/write/delete for actual image files using 'Swift' or similar scalable file storage service.<br />~like aws ami</li>
<li><i><b>Network Management</b>:</i> <b>Quantum/Melange</b><br />Provides all the networking mechanisms required in any instance or environment as a service. Handels network interface cards plug/un-plug actions, ip allocation procedures along-with capability enhancement possible to virtual switches.</li>
<li><i><b>Block Storage</b>: </i><b>Cinder</b><br />Enables to attach volumes for persistent usage. Detach them, snapshot them.<br />~like aws ebs</li>
<li><i><b>WebUI</b>: </i><b>Horizon</b><br />Provides usability improvement for users or projects for managing compute nodes, object storage resources, quota usages and more in a detailed web-app way.<br />~like aws web dashboard</li>
<li><i><b>Authentication</b>: </i><b>Keystone</b><br />Identity management system, providing apis to all other OpenStack components to query for authorization.</li>
<li><b style="font-style: italic;">Billing Service</b><i>: </i><b>Ceilometer</b> (preview)<br />Analyzes quantity, cost-priority and hence billing of all the tasks performed at cloud.</li>
<li><i><b>Cloud Template</b>:</i> <b>Heat</b> (under construction)<br />Build your entire desired cloud setup providing OpenStack a Template for it.<br />~like aws cloudformation</li>
<li><b style="font-style: italic;">OpenStack Common</b><i>: </i><b>OSLO</b> (<i>tenure code</i>)<br />Supposed to contain all common libraries of shared infrastructure code in OpenStack.</li>
</ul>
<br />
Hypervisors are software/firmware/hardware that enables to create, run and monitor virtual machines. OpenStack Compute supports multiple hypervisors like KVM, LXC, QEMU, XEN, VMWARE & more.<br />
<br />
Message Queue Service is used by most of the OpenStack Compute services to communicate with each other using AMQP (Advanced Message Queue Protocol) supporting async calls and callbacks.<br />
<br />
---<br />
<br />
<h3 style="text-align: left;">
<b># Weaving of Components</b></h3>
<br />
<i><b>asciigram:</b> openstack ~ evolution mode, how varied components are connected</i><br />
<div>
~~~~~<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgh0WONdqrDI9t9aIxqWg-8w1iU54TaSMpWsvJCfW3QIckXIqPE95wx-Gs_MciBrAkmDvDUZH2LhNbxl0EB7ExlVMNNc0vAWC5wR88shgwe8Yh5difYlcSZ0xtCXBOfPNvzr0q5MnFrHnA/s1600/openstack_p1_level1.jpg" imageanchor="1" style="clear: left; display: inline !important; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgh0WONdqrDI9t9aIxqWg-8w1iU54TaSMpWsvJCfW3QIckXIqPE95wx-Gs_MciBrAkmDvDUZH2LhNbxl0EB7ExlVMNNc0vAWC5wR88shgwe8Yh5difYlcSZ0xtCXBOfPNvzr0q5MnFrHnA/s1600/openstack_p1_level1.jpg" /></a><br />
~~~~~<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6LfszYX6k2yzpEtnv_Yx1MKDC3akqJFOeXJUcQOcGa_Jfox3EyQaY4QWr_4Rh_gMi2H239gx1aiEUOjI_8YCDWNHIEVmHD68D93mm87Ry7WrRC_Q7YwiYr14Od6QU_diPIV2LoI9_x2A/s1600/openstack_p1_level2.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6LfszYX6k2yzpEtnv_Yx1MKDC3akqJFOeXJUcQOcGa_Jfox3EyQaY4QWr_4Rh_gMi2H239gx1aiEUOjI_8YCDWNHIEVmHD68D93mm87Ry7WrRC_Q7YwiYr14Od6QU_diPIV2LoI9_x2A/s1600/openstack_p1_level2.jpg" /></a></div>
<br /></div>
<div>
<div>
<br class="Apple-interchange-newline" />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
~~~~~<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgn4yGLor41knr5F0JqtF6IZojbJCX5SP3aEfZ2apkINPNeWxuthwxMzyW5jANbegekvqk8b3qL03_tdEqvcl6LcpVe1zXH18fCi4bCn6s4x95NbN0CFdgAwlt_hdlbKubbC4znXxHJe4M/s1600/openstack_p1_level3.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgn4yGLor41knr5F0JqtF6IZojbJCX5SP3aEfZ2apkINPNeWxuthwxMzyW5jANbegekvqk8b3qL03_tdEqvcl6LcpVe1zXH18fCi4bCn6s4x95NbN0CFdgAwlt_hdlbKubbC4znXxHJe4M/s1600/openstack_p1_level3.jpg" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
~~~~~<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDIG3jwKoz8JZE_FMN_EDRBDQWT-AZ735aCX5JXi5FDvpiBTyqGsaMiQU2EBYd5gfL7rb1xCiMRNeMxvo6JLFvzOcpIBP8r4rlQtnDIxKd6zFNTQMZs4ycmEMAhC3Ry1CxU_3mm674QDk/s1600/openstack_p1_level4.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDIG3jwKoz8JZE_FMN_EDRBDQWT-AZ735aCX5JXi5FDvpiBTyqGsaMiQU2EBYd5gfL7rb1xCiMRNeMxvo6JLFvzOcpIBP8r4rlQtnDIxKd6zFNTQMZs4ycmEMAhC3Ry1CxU_3mm674QDk/s1600/openstack_p1_level4.jpg" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
~~~~~<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFtslbpKtupHisoGNhQ7-cPGdgrj8DJJKOScvhW8IaSLRACw2y2RjvVqlCgKE-DMS-ABN4u6pMoWml2c6rYoJ0nQVviJl1kaZylp8hyphenhyphenzJzjVa2vIIseZZrfbXoQE_BEUl_F6b6J8770zM/s1600/openstack_p1_level5.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFtslbpKtupHisoGNhQ7-cPGdgrj8DJJKOScvhW8IaSLRACw2y2RjvVqlCgKE-DMS-ABN4u6pMoWml2c6rYoJ0nQVviJl1kaZylp8hyphenhyphenzJzjVa2vIIseZZrfbXoQE_BEUl_F6b6J8770zM/s1600/openstack_p1_level5.jpg" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br /></div>
</div>
<div>
<br />
<br /></div>
<br />
---<br />
<br />
<h4 style="text-align: left;">
<b><i>More Links:</i></b></h4>
<br />
<ul style="text-align: left;">
<li>(article) <a href="http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/" target="_blank">Folsom Architecture by Ken Pepple</a></li>
<li>(video) Mark Collier on <a href="http://www.youtube.com/watch?v=dclcFiVvO7g" target="_blank">'Overview of OpenStack & Openstack foundation'</a></li>
<li>(video) Sandy Walsh on <a href="http://www.youtube.com/watch?v=bCsw2kkIWyw" target="_blank">'Introduction to OpenStack'</a></li>
<li>(work) <a href="http://launchpad.net/openstack">http://launchpad.net/openstack</a></li>
<li>(wiki) <a href="http://wiki.openstack.org/">http://wiki.openstack.org</a></li>
<li>(join) <a href="http://planet.openstack.org/">http://planet.openstack.org</a></li>
<li>(irc) Freenode #openstack #openstack-dev</li>
<li>RedDward : Database as a Service for OpenStack : <a href="https://github.com/stackforge/reddwarf">https://github.com/stackforge/reddwarf</a></li>
</ul>
</div>
abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com111tag:blogger.com,1999:blog-7154053114439400556.post-9014519880794105072013-02-03T06:35:00.000-08:002013-02-03T06:35:02.637-08:00MessQ : message queue for quickly trying any idea<div dir="ltr" style="text-align: left;" trbidi="on">
Past some time while trying up some set-up based on Message Queue at infrastructure... needed a quick to set-up, localhost friendly, network available Message Queue service to try out ideas.<br />
So here is Mess(age)Q(ueue). Something quickly thrown together. Would work later to get it more performance oriented, good to go with smaller projects.<br />
<br />
@GitHub: <a href="https://github.com/abhishekkr/messQ">https://github.com/abhishekkr/messQ</a><br />
@RubyGems: <a href="https://rubygems.org/gems/messQ">https://rubygems.org/gems/messQ</a><br />
_________________________<br />
<br />
<span style="font-size: large;">A Quick Tryout</span><br />
<br />
<b>[+] Install</b><br />
<b>$</b> gem install messQ --no-ri --no-rdoc<br />
<b>[+] Start Server (starts at 0.0.0.0 on port#5566)</b><br />
<div>
<b>$</b> messQ --start</div>
<div>
<b>[+] Enqueue user-id & home value to the Queue</b></div>
<div>
<b>$</b> messQ -enq $USER</div>
<div>
<b>$</b> messQ --enqueue $HOME</div>
<b>[+] Dequeue 2 values from Queue</b><br />
<div>
<b>$</b> messQ -deq</div>
<div>
<b>$</b> messq --dequeue</div>
<div>
<div>
<b>[+] Stop Server</b></div>
<div>
<b>$</b> messQ --stop</div>
<div>
_________________________<br />
<br />
<span style="font-size: large;">Via Code</span><br />
<b><br /></b>
<b>[+] Install</b><br />
<b>$</b> gem install messQ --no-ri --no-rdoc<br />
or add following to your <b><i>Gemfile</i></b><br />
gem 'messQ'<br />
<b>require 'messQ'</b><br />
<br />
<b>[+] Start Server</b></div>
<div>
MessQ.host = '127.0.0.1' # default is 0.0.0.0</div>
<div>
MessQ.port = 8888 # default is 5566</div>
<div>
MessQ.messQ_server<br />
<div>
<br /></div>
<div>
<b>[+] Enqueue user-id & home value to the Queue</b></div>
<div>
MessQ.host = '127.0.0.1' # default is 0.0.0.0</div>
<div>
MessQ.port = 8888 # default is 5566</div>
MessQ::Agent.enqueue(ENV['USER'])</div>
<div>
MessQ::Agent.enqueue(ENV['HOME'])<br />
<br />
<b>[+] Dequeue 2 values from Queue</b><br />
<div>
MessQ.host = '127.0.0.1' # default is 0.0.0.0</div>
<div>
MessQ.port = 8888 # default is 5566</div>
puts MessQ::Agent.dequeue</div>
<div>
puts MessQ::Agent.dequeue<br />
<div>
<br /></div>
<div>
<div>
<b>[+] Stop Server</b></div>
<div>
MessQ::Server.stop</div>
</div>
</div>
</div>
</div>
abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com0tag:blogger.com,1999:blog-7154053114439400556.post-52643016401681814082012-09-19T11:07:00.001-07:002012-09-19T11:11:04.961-07:00ci-go-nfo v0.0.1 : console util for ThoughtWorks' Go CI Server<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="separator" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em; text-align: center;">
<a href="http://abhishekkr.github.com/ci-go-nfo" target="_blank"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwkp6UraiKmY_UOJhazADwEfZOTlRW0EgOu_3DInN9uusTcrOtpZSsWi4UZgtX8N0OA9a3Vq7WNO7e2jyi7f3dUdxKVFd8TMnLfW4kwfaW51Q087UfgJmDf7xpDIM8XIHGBPgRr3M3D4c/s1600/cigonfo.jpg" /></a></div>
<br />
<a href="http://abhishekkr.github.com/ci-go-nfo" target="_blank"><span style="font-family: "Courier New",Courier,monospace;">ci-go-nfo</span></a> v0.0.1<br />
<br />
<br />
<br />
Just a rubygem console utility to get focussed INFO about your Go Continuous Integration pipeline
easily, no more switching again to browsers.<br />
<br />
@<a href="https://rubygems.org/gems/ci-go-nfo" target="_blank">RubyGems</a>: <a href="https://rubygems.org/gems/ci-go-nfo">https://rubygems.org/gems/ci-go-nfo</a><br />
<br />
@<a href="https://github.com/abhishekkr/ci-go-nfo" target="_blank">GitHub</a>: <a href="https://github.com/abhishekkr/ci-go-nfo">https://github.com/abhishekkr/ci-go-nfo</a><br />
<br />
<br />
<i><b><span style="font-size: large;">Installation</span> </b></i><br />
<br />
<b>$</b> gem install ci-go-nfo<br />
<br />
<br />
<br />
<span style="font-size: large;"><i><b>Usage
Ci-Go-Nfo ver.0.0.1</b></i> </span><br />
<br />
to set-up credential config for your go-ci<br />
$ <b>ci-go-nfo setup</b><br />
it asks for<br />
(a.) the location where you want to store your configuration file<br />
(b.) the URL for your Go Server like http://my.go.server:8153<br />
(c.) then username and password (create a read-only a/c for it) <br />
<br />
<br />
<br />
to show go-ci info of all runs<br />
$ <b>ci-go-nfo</b><br />
<br />
to show go-ci info of failed runs<br />
$ <b>ci-go-nfo fail</b><br />
<br />
to show go-ci info of passed runs<br />
$ <b>ci-go-nfo pass</b><br />
<br />
_____<br />
<br />
.....more to come<br />
<br />
<br />
<span style="font-size: large;">output example: </span><br />
<br />
$ <b>ci-go-nfo setup</b><br />
<div style="background-color: black;">
<span style="color: #999999;"> </span><span style="color: red;"><span style="color: #999999;">Store sensitive Go Configs in file {current file: /home/myuser/.go.abril}: </span><br style="color: #999999;" /><br style="color: #999999;" /><span style="color: #999999;"> Enter Base URL of Go Server {like http://<ip>:8153}: <br /> <span style="color: #cccccc;">http://my.go.server:8153</span></ip></span><br style="color: #999999;" /><br style="color: #999999;" /><span style="color: #999999;"> This is better to be ReadOnly account details...</span><br style="color: #999999;" /><br style="color: #999999;" /><span style="color: #999999;"> Enter Log-in UserName: <span style="color: #cccccc;">go_user</span></span><br style="color: #cccccc;" /><br style="color: #999999;" /><span style="color: #999999;"> Password: <span style="color: #cccccc;">restrictedpassword</span></span></span><b style="color: #cccccc;"><br /></b></div>
<div style="color: #cccccc;">
<br /></div>
$ <b>ci-go-nfo pass</b><br />
<div style="background-color: black;">
<b><span style="color: red;">my_pipeline -> specs -> specs</span></b></div>
<div style="background-color: black;">
<b style="color: lime;">Success</b> <span style="color: #999999;"> for run#</span><span style="color: #6aa84f;">2</span> <span style="color: #f1c232;">at 2012-09-19T04:24:38</span></div>
<div style="background-color: black;">
<span style="color: #999999;">details at</span> <span style="color: #3d85c6;">http://my.go.server:8153/go/tab/build/detail/my_pipeline/10/specs/2/specs</span></div>
<div style="background-color: black;">
<br />
<b style="color: red;">my_pipeline -> package ->gemify</b></div>
<div style="background-color: black;">
<b><span style="color: lime;">Success</span></b> <span style="color: #cccccc;"> f<span style="color: #999999;">or run#</span></span><span style="color: #6aa84f;">1</span> <span style="color: #f1c232;">at 2012-09-19T07:04:39</span></div>
<div style="background-color: black;">
<span style="color: #999999;">details at</span> <span style="color: #3d85c6;">http://my.go.server:8153/go/tab/build/detail/my_pipeline/10/package/1/gemify</span></div>
<br />
$ <b>ci-go-nfo fail</b><br />
<div style="background-color: black;">
<b><span style="color: red;">your_pipeline -> smoke -> cukes</span></b></div>
<div style="background-color: black;">
<b style="color: lime;">Failure</b> <span style="color: #999999;"> for run#</span><span style="color: #6aa84f;">5</span> <span style="color: #f1c232;">at 2012-09-19T04:24:38</span></div>
<div style="background-color: black;">
<span style="color: #999999;">details at</span> <span style="color: #3d85c6;">http://my.go.server:8153/go/tab/build/detail/your_pipeline/7/smoke/5/cukes</span></div>
<br />
$ <b>ci-go-nfo </b><br />
<div style="background-color: black;">
<b><span style="color: red;">my_pipeline -> specs -> specs</span></b></div>
<div style="background-color: black;">
<b style="color: lime;">Success</b> <span style="color: #999999;"> for run#</span><span style="color: #6aa84f;">2</span> <span style="color: #f1c232;">at 2012-09-19T04:24:38</span></div>
<div style="background-color: black;">
<span style="color: #999999;">details at</span> <span style="color: #3d85c6;">http://my.go.server:8153/go/tab/build/detail/my_pipeline/10/specs/2/specs</span></div>
<div style="background-color: black;">
<br />
<b style="color: red;">my_pipeline -> package ->gemify</b></div>
<div style="background-color: black;">
<b><span style="color: lime;">Success</span></b> <span style="color: #cccccc;"> f<span style="color: #999999;">or run#</span></span><span style="color: #6aa84f;">1</span> <span style="color: #f1c232;">at 2012-09-19T07:04:39</span></div>
<div style="background-color: black;">
<span style="color: #999999;">details at</span> <span style="color: #3d85c6;">http://my.go.server:8153/go/tab/build/detail/my_pipeline/10/package/1/gemify</span></div>
<div style="background-color: black;">
<br /></div>
<div style="background-color: black;">
<b><span style="color: red;">your_pipeline -> smoke -> cukes</span></b>
<b style="color: lime;">Failure</b> <span style="color: #999999;"> for run#</span><span style="color: #6aa84f;">5</span> <span style="color: #f1c232;">at 2012-09-19T04:24:38</span>
<span style="color: #999999;">details at</span> <span style="color: #3d85c6;">http://my.go.server:8153/go/tab/build/detail/your_pipeline/7/smoke/5/cukes</span></div>
</div>
abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com1tag:blogger.com,1999:blog-7154053114439400556.post-45678619043743826942012-08-05T05:26:00.002-07:002013-12-22T00:06:46.233-08:00Puppet ~ a beginners concept guide (Part 3) ~ Modules much more<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div style="font-family: Times,"Times New Roman",serif; text-align: right;">
<span class="Apple-style-span" style="background-color: #fff9ee; color: #222222;"><i>you might prefer first reading guide <a href="http://justfewtuts.blogspot.in/2012/05/puppet-beginners-concept-guide-part-1.html" target="_blank">Part#1</a>(intro to puppet), & <a href="http://justfewtuts.blogspot.in/2012/07/puppet-beginners-concept-guide-part-2.html" target="_blank">Part#2</a>(intro to modules)</i></span><br />
<i style="color: #222222; font-family: 'Times New Roman';">the section after this <a href="http://justfewtuts.blogspot.com/2013/07/wip-puppet-beginners-concept-guide-part.html" target="_blank">Part#4(Where is my Data?)</a> discussing how to handle configuration data</i></div>
<span class="Apple-style-span" style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: medium;"><br /></span>
<span class="Apple-style-span" style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: medium;">Puppet</span><br />
<span class="Apple-style-span" style="background-color: #fff9ee; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 15px; line-height: 21px;"><b><span class="Apple-style-span" style="font-size: xx-small;">beginners concept guide</span></b></span><span style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"> (Part <b>3</b>)</span><br />
<br />
<h2 style="text-align: left;">
<b><span style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"><i>Modules with More</i></span></b></h2>
<div>
<span style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">here some time on the practices to prefer while writing most of your modules<i><br /></i></span></div>
<br />
<b style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">[] <i>HowTo Write Good Puppet Modules</i></b><i style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"> </i><span class="Apple-style-span" style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: xx-small; font-style: italic;"><br /><span style="font-size: x-small;">(so everyone could use it and you could use it everywhere)</span></span><br />
<ul>
<li><span style="background-color: white; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"><b>platform-agnostic</b><br />With change in Operating System distro; module also might require difference in package names, configuration file locations, device port names, system commands and more.<br />Obviously, it's not expected to test each and every module against each and every distro and get it full-proof for community usage. But what's expected is to use<span style="font-family: "Courier New",Courier,monospace;"> <i>case $operatingsystem{...}</i></span> statements for whatever distros you can and let the users get notified in case they gotta add something for their distro by<span style="font-family: "Courier New",Courier,monospace;"> <i>fail("")</i></span>, and might also contribute back..... like following </span><br /><span style="font-family: 'Courier New', Courier, monospace;">case $operatingsystem {<br /> centos, redhat: {<br /> $libxml2_development = 'libxml2-devel'<br /> }<br /> ubntu, debian: {<br /> $libxml2_development = 'libxml2-dev'<br /> }<br /> default: {<br /> fail("Unrecognized libxml2 development header package name for your O.S. $operatingsystem")<br /> } <br />}</span><br />
<span style="background-color: white; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">~</span></li>
<li><span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;"><span style="font-size: 15px; line-height: 21px;"><b>untangled puppet strings</b><br />
You are writing puppet modules. Good chance is you have a client or personal environment to manage for which you had a go at it.<br />That means there gonna be your environment specific some client or personal code &/or configuration that is 'for your eyes only'. This will prohibit you from placing any of your module in Community.<br />It's wrong on two main fronts. First, you'll end up using so much from OpenSource and give back nothing. Second, your modules will miss on the community update/comment action.<br />So, untangle all your modules into atomic service level modules. Further modularize those modules into service puppet-ization requirement. That will be like sub-modules for install, configure, service and whatever more you can extract out. Now these sub-modules can be clubbed together to and we can move bottom-up gradually.<br />Now you can just keep your private service modules to yourself, go ahead and use the community trusted and available modules for whatever you can..... try making minor updates to those and also contribute these updates. Write the others that you don't find out in the wild and contribute those too for community to use, update and improve.<br />~<br />
</span></span></li>
<li><span style="background-color: white; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"><b>no data in c~o~d~e</b><br />Now when you are delivering 'configuration as a code', adapt the good coding practices applicable in this domain. One of those is keeping data separate than the code, as in no db-name, db-user-name, db-password, etc. details stored directly in the module's manifest intending to create the db-config file.<br />There will be a detailed section later over different external data usage involving separate <a href="http://docs.puppetlabs.com/guides/parameterized_classes.html#appendix-smart-parameter-defaults" target="_blank">parameter manifest</a> setting up values when included, <a href="http://docs.puppetlabs.com/references/2.6.8/function.html#extlookup" target="_blank">extlookup</a> loading values from CSVs, <a href="http://puppetlabs.com/blog/introducing-puppetdb-put-your-data-to-work/" target="_blank">puppetDB</a>, <a href="http://projects.puppetlabs.com/projects/hiera/" target="_blank">hiera</a> data-store and <a href="http://docs.puppetlabs.com/guides/custom_facts.html" target="_blank">custom facts</a> file to load up key-values.<br />~</span></li>
<li><span style="background-color: white; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"><b>puppet-lint</b><br />To keep the modules adhere to dsl-syntactic correct and beautiful code writing practice. So the DSL and the community contributors, both find it easy to understand your manifests. It's suggested to have it added to rake default of your project to check all the manifests, ran before every repo-check-in.<br />~</span></li>
<li><span style="background-color: white; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"><b>do-undo-redo</b><br />It's suggested to have a undo-manifest ready for all the changes made by a module. It mainly comes in handy for infrastructures/situations where creating and destroying a node is not under your administrative tasks or consumes hell lot of time.<br />Obviously, in case getting new node is easier..... that's the way to go instead of wasting time in undo-ing all the changes (and also relying on that).<br /><b><i>Those are just there for the dry-days when there is no 'cloud'.</i></b><br />~</span></li>
</ul>
<br />
<br />
<br />
<b style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">[] <i>More about Modules</i></b><i style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"> </i><span class="Apple-style-span" style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: xx-small; font-style: italic;">(moreover.....)</span><br />
<div style="text-align: left;">
<span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;"><span style="font-size: 15px; line-height: 21px;">Where to get new:</span></span><span style="font-family: Arial, Helvetica, sans-serif;"><a href="http://forge.puppetlabs.com/">http://forge.puppetlabs.com/</a> is the community-popular home for most of the Puppet Modules.</span></div>
<span style="color: #222222;"><span style="line-height: 21px;"><span style="font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px;">Where to contribute:</span><br /><span style="font-family: Arial, Helvetica, sans-serif;">Can manage your public module at <a href="https://github.com/">GitHub</a> or similar online free repository like </span></span></span><span style="font-family: Arial, Helvetica, sans-serif;"><a href="https://github.com/puppetlabs/puppetlabs-kvm">puppetlabs-kvm</a>.<br />Then you can push your module to <a href="http://docs.puppetlabs.com/puppet/2.7/reference/modules_publishing.html">forge.puppetlabs.com</a>.</span><br />
<div>
<span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;"><span style="font-size: 15px; line-height: 21px;"><br /></span></span></div>
</div>
abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com0tag:blogger.com,1999:blog-7154053114439400556.post-39302351475190327642012-07-28T06:45:00.001-07:002012-07-28T06:45:54.209-07:00DevOps AND 12FactorApp ~ some obsolete & much valid<div dir="ltr" style="text-align: left;" trbidi="on">
<b style="background-color: white;">Why?</b><br />
<div style="text-align: left;">
<span style="font-family: 'Trebuchet MS', sans-serif; font-size: x-small;"><i>Few months ago I came across <a href="http://www.12factor.net/" target="_blank">The Twelve-Factor-App</a> preaching the best practices for building and delivering software. Nothing really new, but a good central place with many good practices for people to refer and compare. </i><i>Recently I saw some implementation of it in an environment where the basic concerns were already handled and thus the solution implemented was redundant and extra cost. To some level also low-grade.</i></span></div>
<br />
<br />
<b style="background-color: white;">What?</b><br />
<br />
Actually what <a href="http://www.12factor.net/" target="_blank">12FactorApp</a> is... it is a good set of ideas around basic set of concerns. The concerns are right, the solutions suggested are situational and the situation is the default/basic setup. With the teams following good DevOps-y practices, they don't turn out to be exactly same.<br />
<br />
So to avoid the confusions for more people and foremost saving me the pain of explaining myself at different places in different times for my views against <a href="https://speakerdeck.com/u/kennethreitz/p/the-12-factor-app" target="_blank">12FactorApp</a>..... here is what the concerns are and what the solutions turn into when following a proper DevOps-y approach.<br />
<br />
<br class="Apple-interchange-newline" /><br />
<br />
<h3>
<b>What @<span style="background-color: white;">12FactorApp doesn't suit at all for DevOps-y Solutions</span></b></h3>
<ol>
<li><span style="background-color: white;">~</span></li>
<li><span style="background-color: white;">Dependencies</span><br /><b style="font-style: italic;">[+] Obsolete: </b>'If the app needs to shell out to a system tool, that tool should be vendored into the app.'<br /><i><b>Changed-to: </b></i><i>Make your automation configuration management system handle it.</i></li>
<li><span style="background-color: white;"><i>Configurations.</i><br /><i><b>[+] Obsolete:</b></i> The twelve-factor app stores config in environment variables changing between deploys w/o changing any code.<br /><span style="font-style: italic;"><b>Changed-to:</b> </span>This is not a fine scalable with disaster management based solution. <span style="background-color: white;">N</span>ow configuration management handles the node level deterministic state. The non-developer box level configuration is no more in code.<br /><i><b>[+] Obsolete:</b></i> The twelve-factor app stores config in environment variables changing between deploys w/o changing any code.<span style="font-style: italic;"><br /><i><b>Changed-to:</b> </i><span style="font-style: normal;">N</span></span></span>ow configuration management handles the node level deterministic state. In such a case keeping configurations in a file is much more verifiable, cleaner and broadly available solution. So, there will be no more noise of different environment level configurations in the same configuration file.</li>
<li><i>~</i></li>
<li>Build, Release, Run<br /><i><b>[+] Obsolete:</b></i> The resulting <i>release</i> contains both the build and config.<br /><b><i>Changed-to:</i></b> Packaging configuration along-with build makes it dependent of a set environment. Any disaster resistant or scalable architecture would be crippled with it as it requires creating new packages every change. Make your automated configuration management solution intelligent enough to infer required configuration and deploy the build.</li>
<li>~</li>
<li>~</li>
<li>Concurrency<br /><i><b>[+] Obsolete:</b></i> Twelve-factor app processes should never daemonize or write PIDfiles.<br />Changed-to: PID files help some automated configuration management solutions to easily identify the 'service' check placed in them. There are operating system level process managers also supporting PIDfiles. Having a pidfile eases up lots of other custom monitoring plug-ins too... and is not a bad practice to have.</li>
<li>~</li>
<li>~</li>
<li>~</li>
<li>~</li>
</ol>
<br />
<br />
<h3 style="text-align: left;">
<b><span style="background-color: white;">Cumulative</span> Correct Concerns 3C@<span style="background-color: white;">12FactorApp and DevOps-y Solutions</span></b></h3>
<span style="background-color: white;">Overall aiming to achieve a </span><span style="background-color: white;"><b style="font-style: italic;">easy-to-setup, clean-to-configure, quick-to-scale</b> and <b style="font-style: italic;">smooth-to-update</b> software development ambiance.</span><br />
<span style="background-color: white;"><b>The 12 Concerns+Solutions:</b></span><br />
<ol style="text-align: left;">
<li>Problem: <b>Maintaining Application Source Code<br />Solution:<br />a.</b> Using Version Control Mechanism, if possible Distributed VCS like <a href="http://git-scm.com/" target="_blank">git</a>. Private hosted (at least private account) code repository.<br /><b>b.</b> Unique application~to~repository mapping i.e. single application or independent library's source code in a single repository.<br /><b>c.</b> For different versions of same application depend on different commit-stages (not even branches in general cases) of the same code repository.<br /></li>
<li>Problem: <b>Managing Application Dependencies<br />Solution:<br />a.</b> Never manually source compile any dependent library or application. Always depend on the standard PackageManager for the intended platform (like <a href="http://www.rpm.org/" target="_blank">rpm</a>, <a href="http://www.debian.org/doc/manuals/debian-faq/ch-pkg_basics.en.html" target="_blank">pkg</a>, <a href="http://rubygems.org/" target="_blank">gem</a>, <a href="http://peak.telecommunity.com/DevCenter/PythonEggs#building-eggs" target="_blank">egg</a>, <a href="http://npmjs.org/" target="_blank">npm</a>). If there are no packages available, create one. It's not really difficult. On a standard practice, I'd suggest to utilize something like <a href="https://github.com/jordansissel/fpm/wiki/" target="_blank">FPM</a> (may be even <a href="https://github.com/bernd/fpm-cookery/blob/master/README.md" target="_blank">fpm-cookery gem</a> if you like), which would give you elasticity of easily changing your platform without worrying for the re-creation of packages. Even creating rpm, gem and other is not too much pain compared to the stability it would bring to infrastructure set-up.<br /><b>b.</b> Make your automated configuration management utility ensure all the required dependencies of your application are pre-installed in correct order of correct version with correct required configurations.<br /><b>c.</b> The dependency configuration will be specific enough to ensure the usage of the installed & configured dependencies. So in case of compiling binary, use static library linking. If you are loading external libraries, ensure the fixated path. Same configuration management tool can be run even in solo/masterless (no-server) usage mode.<br /></li>
<li><b><span style="font-weight: normal;">Problem: </span>Configuration in Code, Configuration at all Deploy</b><b><br />Solution:<br />a.</b> Ideally no configuration details as in node's IP/Name, credentials, etc. shall not be a part of application's codebase. As if such a configuration file is locally available in developer-box repository, in non-alert & non-gitignore days it might get committed to your repository.<br /><b>b.</b> Make your automated configuration management tool generate all these configuration files for a node based on the node-specific details provided to configuration management tool, no the application.<br /><b>c.</b> Suggested practice for keeping these configurations with configuration management tool, also require to utilize a proper different data-store from normal configuration statements. Could be in <a href="http://en.wikipedia.org/wiki/Comma-separated_values" target="_blank">CSV</a>s, <a href="http://projects.puppetlabs.com/projects/hiera/" target="_blank">Hiera</a>, dedicated <a href="https://github.com/example42/puppet-modules/blob/master/cron/manifests/params.pp" target="_blank">parameter</a>'s manifest for a tool like <a href="http://puppetlabs.com/puppet/what-is-puppet/" target="_blank">Puppet</a>. For a tool like <a href="http://www.opscode.com/" target="_blank">OpsCode</a>'s <a href="http://www.opscode.com/chef/" target="_blank">Chef</a>, there is already <a href="http://wiki.opscode.com/display/chef/Data+Bags" target="_blank">databag</a> facility available. Wherever available and required the info should be <a href="http://wiki.opscode.com/display/chef/Encrypted+Data+Bags" target="_blank">encrypted</a> with a non-repository available secret key.<br /></li>
<li><b><span style="font-weight: normal;">Problem: </span>Backing Services</b><b><br />Solution:<br />a.</b> Whatever other application services are required by application to serve can be included in the '<i>Backing Services</i>' list. It will be services like data-stores (<i><a href="http://en.wikipedia.org/wiki/Active_record_pattern" target="_blank">databases</a>, <a href="http://api.rubyonrails.org/classes/ActiveSupport/Cache.html" target="_blank">memory cache</a> and more <a href="http://as.rubyonrails.org/" target="_blank">activesupport</a></i>), smtp services, etc.<br /><b>b.</b> Every information required for these backing services should be configuration details like node-name, credentials, port#, etc. and maintained as a loaded configuration file via configuration management tool.<br /><b>c.</b> If it's a highly complex applications broken into several component applications supporting each other, then all other component applications for any component application are also 'Backing Services'.<br /></li>
<li><b><span style="font-weight: normal;">Problem: </span>Build, Release, Run</b><b><br />Solution:<br />a.</b> The development stage code gets pushed to codebase and after passing intended tests should be pushed to Build Stage for preparing deploy-able (compile, include dependencies) code. Should read <a href="http://en.wikipedia.org/wiki/Continuous_integration" target="_blank">Continuous</a> <a href="http://martinfowler.com/articles/continuousIntegration.html" target="_blank">Integration</a> process for the better approach at it.<br /><b>b.</b> The deploy-able code is packaged ready to deliver in Release Stage and pushed in to the package-manager repositories. The required configuration for execution environment is provided to automated configuration management solution.<br /><b>c.</b> Run Stage involves release application package from package-manager and intended system-level configurations via configuration management solution.<br /></li>
<li><b><span style="font-weight: normal;">Problem: </span>Processes</b><b><br />Solution:<br />a. </b>No persistent data related to application shall be kept along-with it. All of user-input & calculated information required for service shall be at the '<i>Backing Services</i>' available to all the instances of the application of that environment. Helping the application to be stateless.<br /><b>b.</b> Get the static assets compiled at '<i>Build Stage</i>', served via <a href="http://en.wikipedia.org/wiki/Content_delivery_network" target="_blank">CDN</a> and cached at <a href="http://haproxy.1wt.eu/" target="_blank">load balancing server</a>.<br /><b>c.</b> Session state data is a good candidate to be stored and retrieved using backing memory powered cache service (like memcache or redis) providing full-blown stateless servers where lossing/killing one and bringing another doesn't impact on user experience.<br /></li>
<li><b><span style="font-weight: normal;">Problem: </span>Port Binding</b><b><br />Solution:<br />a.</b> Applications shouldn't allow any run-time injection to get utilized by 'Backing Services' but instead expose their interaction over any <a href="http://en.wikipedia.org/wiki/Representational_state_transfer" target="_blank">RESTful</a> (or similar) protocol. <br /><b>b.</b> In a standard setup, the data/information store/provider opens up a socket and the retriever contacts at the socket with required data transaction protocol. Now this data/information provider can be 'Backing Service' (like db service) or could be the primary application providing information over to a 'Backing Service' (like application server, load balancer).<br /><b>c.</b> Either way, they get configured with primary application via automated configuration management by url, port and any other service specific required detail being provided.<br /></li>
<li><b><span style="font-weight: normal;">Problem: </span>Concurrency</b><b><br />Solution:<br />a.</b> Here concurrency is mainly used for highly scalable (on requirement) process model, which is almost equivalent to how used libraries manage internal concurrent processes.<br /><b>b.</b> All application & '<i>Backing Service</i>' processes should be such managed that process count of one doesn't effect another as in say access via load balancer over multiple http processes.<br /><b>c.</b> All the processes have a process-type and process-count. There should be a process manager to handle continuous execution of that process with that count. Now it could be ruby rack server to be run with multiple threads on same server, or multiple nodes with nginx serving indecent amount of users via a load balancer.<br /></li>
<li><b><span style="font-weight: normal;">Problem: </span>Disposability</b><b><br />Solution:<br />a. </b>Quick code & configuration deployment. Configuration Management solution makes sure of the latest (or required stage) application code & configuration changes cleanly & quickly replace the old application exactly as desired.<br /><b>b.</b> Application (and '<i>Backing Services</i>') Architecture <a href="http://labs.vmware.com/publications/elastic-systems" target="_blank">shall be</a> <a href="http://www.slideshare.net/ericvh/scalable-elastic-systems-architecture-sesa" target="_blank">elastic</a>, spawning up new nodes under a load-balancer and destroying when the process-load is less must be smooth.<br /><b>c.</b> Application's data transactions & task list should be <a href="http://lwn.net/Articles/191059/" target="_blank">crash-proof</a>. The data & tasks shall be managed to handle the re-schedule of those processes in case of application crash.<br /></li>
<li><b><span style="font-weight: normal;">Problem: </span>Dev/Prod Parity</b><b><br />Solution:<br />a.</b> Keep dev, staging, ..., and production environments as similar as possible.If not in process count and machine nodes count, but necessarily similar on the deployment tasks. Could utilize '<a href="http://vagrantup.com/" target="_blank">vagrant</a>' in coordination with configuration management solution to get quick production like environments at any development box.<br /><b>b.</b> Code manages the application and configuration both, any developer (with considerable system level expertise) could get a hang of configuration management frameworks and manage them. Using '<i>Backing Services</i>' as mentioned would enable environment based different service providers.<br /><b>c.</b> Adapting <a href="http://www.slideshare.net/jmcgarr/continuous-delivery-8341276" target="_blank">Continuous</a> <a href="http://en.wikipedia.org/wiki/Continuous_deployment" target="_blank">Delivery</a> would also ensure no new change in code or configuration breaks the deployment.<br /></li>
<li><b><span style="font-weight: normal;">Problem: </span>Logs</b><b><br />Solution:</b><b>a.</b> All staging/production environment will have application and 'Backing Services' promoting its logs to a central (like <a href="http://www.syslog.org/" target="_blank">syslog</a>, <a href="http://www.balabit.com/network-security/syslog-ng" target="_blank">syslog-ng</a>, <a href="https://github.com/logstash/logstash" target="_blank">logstash</a>, etc) log hub for archival, if required back-up proof. It can be queried here for analyzing trnds in application performance over past time.<b><br />b.</b> The central log solution is not configured within applications but the log solution takes care of what to pick and collect, can even have a look at log routers (<a href="https://github.com/fluent/fluentd" target="_blank">fluentd</a>, <a href="https://github.com/heroku/logplex" target="_blank">logplex</a>, <a href="http://www.rsyslog.com/" target="_blank">rsyslog</a>).<br /><b>c.</b> Specific log trends can be put to alert everyone effected whenever captured again at Central Log Services (like <a href="http://www.graylog2.org/about" target="_blank">graylog2</a>, <a href="http://www.splunk.com/" target="_blank">splunk</a>, etc).<br /></li>
<li><b><span style="font-weight: normal;">Problem: </span>Admin Processes</b><b><br />Solution:<br />a.</b> Application level admin processes (like db-migration, specific-case-tasks, debug console, etc.) shall also pick the same code and configuration as the running instances of application.<br /><b>b.</b> The admin tasks script related to application shall also ship with application code and evolve with it. As <a href="http://guides.rubyonrails.org/migrations.html" target="_blank">db management</a> rake tasks in <a href="http://rubyonrails.org/" target="_blank">RubyOnRails</a> applications, run using '<a href="http://gembundler.com/" target="_blank">bundler</a>' to stay <a href="http://asciicasts.com/episodes/201-bundler" target="_blank">pick required Environment</a> related <a href="http://yehudakatz.com/2010/04/21/named-gem-environments-and-bundler/" target="_blank">library versions</a>.<br /><b>c.</b> Languages with <a href="http://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop" target="_blank">REPL</a> shell (like python) or providing it via separate utility (like '<a href="http://guides.rubyonrails.org/command_line.html" target="_blank">rails console</a>' for rails) give an advantage to easily debug an environment specific issue (which might me arising due to library versions of that environment, data in-consistency, etc) by directly playing around with the objects seemingly acting as the source of flaw.</li>
</ol>
<br />
<b><br /></b><br />
<h3 style="text-align: left;">
<b>As I Always Say</b></h3>
<b>Every Generic Solution is very Specifically Placed.</b><br />
<span style="background-color: white;"><br /></span><br />
<span style="background-color: white;"><br /></span></div>abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com0tag:blogger.com,1999:blog-7154053114439400556.post-38454697582581421002012-07-17T13:16:00.000-07:002012-07-17T13:16:47.507-07:00how to host your Public YUM (or any) Repo<div dir="ltr" style="text-align: left;" trbidi="on">
almost an year ago came up the simple idea of getting a really simple static-content (html,css,js,...) website on public portal hosted by Google AppEngine for free upto a daily revived usage scheme: <a href="http://gae-flat-web.appspot.com/">http://gae-flat-web.appspot.com/</a><div>
<br /></div>
<div>
few days back I was just playing around custom yum repos and thought why not get up one of my own for public usage with RPMs served for either my projects or other non-available rpms, and what I came up with is: <a href="http://yum-my.appspot.com/flat_web/index.htm">http://yum-my.appspot.com/flat_web/index.htm</a></div>
<div>
<br /></div>
<div>
it's nothing fascinating but just a re-mixed usage of project created from <b><i><a href="http://gae-flat-web.appspot.com/" target="_blank">gae-flat-web</a></i></b>.</div>
<div>
<br /></div>
<div>
you can access base skeleton of this re-mixed gae-yum-my (the easy way to host your yum repo online) at <a href="https://github.com/abhishekkr/gae-yum-my">https://github.com/abhishekkr/gae-yum-my</a> which also has rpm served for gae-flat-web.</div>
<div>
<br /></div>
<div>
now to see how you could get one too~<br /><br />First Task, register a new portal on <a href="http://appengine.google.com/" target="_blank">Google AppEngine</a> (it's free for reasonable limited usage)using your Google Account. Say, your appengine portal is name MY-YUM-MY.</div>
<div>
<br /></div>
<div>
Now the fun begins.<br /><br />cloned the starter kit<br /><blockquote class="tr_bq">
<b>$ git</b> clone</blockquote>
</div>
<div>
enter the cloned repo</div>
<blockquote class="tr_bq">
<b>$ cd</b> gae-yum-my</blockquote>
<div>
to actually change your application name in app.yaml</div>
<blockquote class="tr_bq">
<b>$ sed</b> -i 's/yum-my/<span style="background-color: white;">MY-YUM-MY/g' app.yaml</span></blockquote>
<div>
create the required linux distro, release branch</div>
<blockquote class="tr_bq">
<b>$ mkdir</b> yummy/<distro><releasever>/<basearch<span style="background-color: white;">></span></blockquote>
<div>
copy all required RPMs in that distro, release branch</div>
<blockquote class="tr_bq">
<b>$ cp</b> <ALL_MY_RPMS_of__DISTRO_ReleaseVer_BaseArch> yummy/<distro><releasever>/<basearch>/</blockquote>
<div>
prepare yum-repo-listing using createrepo command</div>
<blockquote class="tr_bq">
<b>$ createrepo</b> yummy/<distro><releasever>/<basearch>/ </blockquote>
<div>
now, place a file 'flat_web/yum-my-el6<or-whichever>.repo' with content </div>
<blockquote class="tr_bq">
<span style="background-color: white;">[yum-my-</span><<span style="background-color: white;">distro</span>><<span style="background-color: white;">releasever</span>><span style="background-color: white;">-</span><<span style="background-color: white;">basearch</span>><span style="background-color: white;">]</span><span style="background-color: white;"> </span></blockquote>
<blockquote class="tr_bq">
name=MY-YUM-MY<span style="background-color: white;"> </span></blockquote>
<blockquote class="tr_bq">
baseurl=<a href="http://youryummy.appspot.com/yummy/%3Cdistro%3E$releasever/$basearch">http://MY-YUM-MY.appspot.com/yummy/<span style="color: black;"><</span>distro<span style="color: black;">></span>$releasever/$basearch</a><span style="background-color: white;"> </span></blockquote>
<blockquote class="tr_bq">
<a href="http://youryummy.appspot.com/yummy/%3Cdistro%3E$releasever/$basearch"></a>enabled=1<span style="background-color: white;"> </span></blockquote>
<blockquote class="tr_bq">
gpgcheck=0</blockquote>
<div>
<br /></div>
<div>
and can link this file on your 'flat_web/index.htm' homepage </div>
<div>
<br /></div>
<div>
to host: </div>
<blockquote class="tr_bq">
$ <google_appengine_path>/<b>appcfg.py</b> update <MY-YUM-MY_path></blockquote>
<div>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</div>
<div>
<br /></div>
<div>
now you yum repo has a homepage at http://MY-YUM-MY.appspot.com</div>
<div>
<br /></div>
<div>
and placing tthe *.repo file above with hinted content will get the RPMs added to you repo accessible.</div>
</div>abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com0tag:blogger.com,1999:blog-7154053114439400556.post-30173790834752345442012-07-08T05:32:00.000-07:002013-03-13T04:29:53.519-07:00Puppet ~ a beginners concept guide (Part 2) ~ road to Modules<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: right;">
<span class="Apple-style-span" style="background-color: #fff9ee; color: #222222;"><i><span style="font-family: Times, 'Times New Roman', serif;">you might prefer first reading<a href="http://www.blogger.com/goog_1050588259"> </a></span><a href="http://justfewtuts.blogspot.in/2012/05/puppet-beginners-concept-guide-part-1.html" target="_blank"><span style="font-family: 'Trebuchet MS', sans-serif;">[ Puppet ~ beginners' concept guide ]</span><span style="font-family: Times, 'Times New Roman', serif;"> Part#1</span></a><br />the section after this <a href="http://justfewtuts.blogspot.com/2012/08/puppet-beginners-concept-guide-part-3.html">Part#3 talks about More on Modules is you work serious</a></i></span></div>
<span class="Apple-style-span" style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: medium;"><br /></span>
<span class="Apple-style-span" style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: medium;">Puppet</span><br />
<span class="Apple-style-span" style="background-color: #fff9ee; color: #222222; font-family: 'Courier New', Courier, monospace; font-size: 15px; line-height: 21px;"><b><span class="Apple-style-span" style="font-size: xx-small;">beginners concept guide</span></b></span><span style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"> (Part <b>2</b>)</span><br />
<span style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"><i>The Road To Modules</i></span><br />
<span style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"><i><br /></i></span>
<b style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">[] <i>Puppet Modules?</i></b><i style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"> </i><span class="Apple-style-span" style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: xx-small; font-style: italic;">(no, no..... nothing different conceptually)</span><br />
<span style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">Puppet Modules (<i>like in most other technological references, and the previous part of this tutorial</i>) are libraries to be loaded and shared as per the required set of configuration.</span><br />
<span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;"><span style="font-size: 15px; line-height: 21px;"><br /></span></span>
<span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;"><span style="font-size: 15px; line-height: 21px;">Think if you have a war application to be deployed over tomcat. For the stated requirement you require tomcat to be present on the machine with correct required configurations and war file to be correctly downloaded and placed on the machine with correct permissions.</span></span><br />
<span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;"><span style="font-size: 15px; line-height: 21px;">In a general scenario requirement </span></span><span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">like this, </span><span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;"><span style="font-size: 15px; line-height: 21px;">two modules come up. One to install, configure and service-start tomcat service. Another to <i>download/locate war file</i>, use <i>tomcat's configure</i> and <i>service sub-module</i>.</span></span><br />
<br />
<b style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">[] <i>Logic of Structure Logic</i></b><i style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"> </i><span class="Apple-style-span" style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: xx-small; font-style: italic;">(just how is your module structured and )</span><br />
<span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;"><span style="font-size: 15px; line-height: 21px;">The different components of structural design followed by each puppet module:</span></span><br />
<ul style="text-align: left;">
<li><span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;"><span style="font-size: 15px; line-height: 21px;"><b>manifests</b> <br />All your '<b><module/submodule>.pp</b>' manifest files go into '<b><module_dir>/manifests</b>'. <br />Puppet has an auto-load service for modules/sub-modules, so the naming of these <b>*.pp</b> files should be suiting the class names.<br />As discussed above for a 'tomcat' module, you are also gonna create sub-modules like '<b>tomcat::install</b>', '<b>tomcat::configure</b>', and '<b>tomcat::service</b>'. <br />So the files that will get create be '<b><tomcat-module>/manifests/install.pp</b>', '</span></span><span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"><b><tomcat-module>/manifests/configure.pp</b>', '</span><span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"><b><tomcat-module>/manifests/service.pp</b>'.<br />Now if there would have been a sub-module like '<b>tomcat::configure::war</b>', then the file-path would go like '</span><span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"><b><tomcat-module>/manifests/configure/war.pp</b>'.</span></li>
<li><span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;"><span style="font-size: 15px; line-height: 21px;"><b>templates</b><br />As any other language, where you want some static data merged with varying passed-on or environment </span></span><span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"> variables and pushed in somewhere as content. Say, for '<i>tomcat::config</i>' sub-module as you wanna parameter-ize somethings like '<i>war</i>' file name. Then this war file-name is being passed-on by '<i>deploy_war</i>' module.<br />This ruby template goes in</span><span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"> '</span><span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"><b><tomcat-module>/files/war_app.conf.erb</b>' and whenever required it's content received as "<b>template('<tomcat-module>/war_app.conf.erb')</b>"</span></li>
<li><span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;"><span style="font-size: 15px; line-height: 21px;"><b>files</b><br />Any kin'of static file can be served from a module using puppet's fileserver mount points. Every puppet module has a default file-server mount location at '<b><tomcat-module>/files</b>'.<br />So a file like '<b><tomcat-module>/files/web.war</b>' get to be served at Puppet Agents pointing to source of '<b>puppet:///modules/<tomcat-module>/web.war</b>'.</span></span></li>
<li><span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;"><span style="font-size: 15px; line-height: 21px;"><b>lib</b><br />This is the place where you can plug-in your custom mods to puppet and use your newly powered up puppet features.<br />This is the one feature that lets you actually utilize your ruby-power and <b>add-on custom facts, providers & types</b> (with default location <span style="font-weight: normal;">at '</span><b style="font-weight: bold;"><tomcat-module> /lib/ <facter|puppet></b><span style="font-weight: normal;">', </span><b><span style="font-weight: normal;">'</span><b><tomcat-module> /lib/puppet/ <parser|provider|type></b><span style="font-weight: normal;">'</span>)</b> to be used via puppet in your modules. To be used it requires '<b>pluginsync = true</b>' configuration to be present at '<b>puppet.conf</b>' level.<br />We'll discuss this in more detail with all sorts of examples in following up blogs and add the links here. Until then it can be referred at <a href="http://docs.puppetlabs.com/guides/plugins_in_modules.html" target="_blank">docs.puppetlabs.com</a>.</span></span></li>
<li><span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;"><span style="font-size: 15px; line-height: 21px;"><b>spec/tests</b><br />As <b>Love</b> needs <b>Money</b> to avoid worldly issues affect its charm<b>. </b>Similarly, <b>Code</b> need <b>Tests.</b> In location </span></span><b style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"><span style="font-weight: normal;">'</span><b><tomcat-module>/spec/</b></b><span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">' you can have your <a href="http://rspec-puppet.com/" target="_blank">puppet-rspec</a> <b><a href="http://bombasticmonkey.com/2012/03/02/automatically-test-your-puppet-modules-with-travis-ci/" target="_blank">tests</a></b> for puppet module.<br /><span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: small; line-height: normal;"><span style="font-size: 15px; line-height: 21px;">The path </span></span><b><span style="font-weight: normal;">'</span><b><tomcat-module>/tests/</b></b>' would have common examples on how the module classes would be defined.</span></li>
</ul>
<br />
<br />
<br />
<b style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;">[] <i>Modules Fundamental Live</i></b><i style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: 15px; line-height: 21px;"> </i><span class="Apple-style-span" style="background-color: #fff9ee; color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif; font-size: xx-small; font-style: italic;">(mean the actual code sample.....)</span><br />
<span style="color: #222222; font-family: Georgia, Utopia, 'Palatino Linotype', Palatino, serif;"><span style="font-size: 15px; line-height: 21px;"><br /></span></span>
<br />
<div>
<script src="https://gist.github.com/abhishekkr/3070758.js"></script></div>
</div>abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com0tag:blogger.com,1999:blog-7154053114439400556.post-80981578399156340952012-06-08T09:46:00.000-07:002012-06-08T09:56:23.832-07:00rss-motor v0.0.4 ~ a rubygem to ease up your RSS interactions<div dir="ltr" style="text-align: left;" trbidi="on">
started a new rubygem project <b><i><a href="http://rubygems.org/gems/rss-motor" target="_blank">'rss-motor'</a> (</i></b><a href="http://rubygems.org/gems/rss-motor">http://rubygems.org/gems/rss-motor</a><b><i>)</i></b> to aid all RSS consuming code by providing the entire (or filtered as per choice) of feeds as Array of Hash values per Feed Item.<br />
<pre style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; font-family: 'Bitstream Vera Sans Mono', Courier, monospace; font-size: 17px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: -webkit-auto; white-space: pre-wrap;"><span style="color: #333333;">===============================================
</span><span style="color: #351c75;"> ||}} //\ //\ _ ||\/|| ||@|| ~++~ ||@|| ||))
||\\ _\\ _\\ || || ||_|| || ||_|| ||\\</span><span style="color: #333333;">
===============================================</span></pre>
<br />
I tried it in a new project <a href="https://github.com/abhishekkr/rss-fanatic" target="_blank"><b><i>'rss-fanatic'</i></b></a> <b><i>(</i></b><a href="https://github.com/abhishekkr/rss-fanatic">https://github.com/abhishekkr/rss-fanatic</a><b><i>)</i></b> making to help out RSS Feed fanatics collecting required content without pain of browsing/saving/downloading. <i><span style="font-size: x-small;">Though <a href="https://github.com/abhishekkr/rss-fanatic" target="_blank">RSS-Fanatic</a> project is just started and shall be usable in some time soon</span></i>.<br />
<pre style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #333333; font-family: 'Bitstream Vera Sans Mono', Courier, monospace; font-size: 17px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: -webkit-auto; white-space: pre-wrap;"></pre>
<pre style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #333333; font-family: 'Bitstream Vera Sans Mono', Courier, monospace; font-size: 17px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: -webkit-auto; white-space: pre-wrap;"></pre>
<div>
<br /></div>
<div>
Here is just a mini HowTo easily power you code with <b>rss-motor</b>:</div>
<div>
<br /></div>
<div>
<span style="font-size: large;"><b>First</b></span>, obviously you'll need to install the gem</div>
<blockquote class="tr_bq">
$ <span style="color: #cc0000; font-family: 'Courier New', Courier, monospace;">gem install rss-motor</span></blockquote>
<div>
or if luckily you already have a Gemfile usage, add following lines to it</div>
<div>
<pre style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;"><blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;"><span class="n" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">source</span> <span class="s2" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #dd1144; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">"http://rubygems.org"</span>
<span class="n" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">gem</span> <span class="s1" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #dd1144; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">'rss-motor'</span></span></blockquote>
<span style="font-family: 'Times New Roman'; white-space: normal;"><pre style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<span style="font-size: large;"><b>Now</b></span>, currently available engines from the rss-motor</span></pre>
<ul style="text-align: left;">
<li>simple way of getting all the items as array of key=value<br /><span style="font-family: 'Courier New', Courier, monospace;">puts <span style="color: #cc0000;">Rss::Motor.rss_items</span> 'http://news.ycombinator.com/rss'</span></li>
</ul>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span></div>
<ul style="text-align: left;">
<li>get an array of items filtered by one or more keywords<br /><span style="font-family: 'Courier New', Courier, monospace;">puts "#{<span style="color: #cc0000;">Rss::Motor.rss_grep</span> 'http://news.ycombinator.com/rss', ['ruby', 'android']}"</span></li>
</ul>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span></div>
<ul style="text-align: left;">
<li>to filter even the data of content available at >link/< field present in item plus normal filter<br /><span style="font-family: 'Courier New', Courier, monospace;">puts "#{<span style="color: #cc0000;">Rss::Motor.rss_grep_link</span> 'http://news.ycombinator.com/rss', ['ruby', 'android']}"</span></li>
</ul>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span></div>
<div style="text-align: left;">
<i>now go on, ride your own rss-bikes.....</i></div>
</div>
</div>abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com0tag:blogger.com,1999:blog-7154053114439400556.post-44355331426854764532012-06-06T13:57:00.002-07:002012-06-06T14:01:32.615-07:00Get Set Go Lang ~ part#1<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: center;">
<a href="http://golang.org/doc/gopher/frontpage.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="100" src="http://golang.org/doc/gopher/frontpage.png" width="73" /></a></div>
<div style="text-align: right;">
<span style="font-size: large;"><i>Get Set</i> <span style="font-family: Verdana, sans-serif;"><b><i>GO</i></b></span> Lang</span></div>
part# <b><span style="font-family: 'Courier New', Courier, monospace;">1</span></b><br />
_________________________<br />
<br />
<b>What Is </b>Go <b>Lang?</b> <br />
<i>(in case you just came here while curious web surfing)</i><br />
<i><br /></i><br />
<a href="http://golang.org/" target="_blank">Go</a> is an OpenSource programming platform developed by <a href="http://google.com/" target="_blank">Google</a> (and <a href="http://golang.org/CONTRIBUTORS" target="_blank">contributors</a>) to be expressive and efficient at the same point.<br />
It's distributed under <a href="http://golang.org/LICENSE" style="font-style: italic;" target="_blank">BSD-<b>style</b> License</a><br />
It's a concurrency favoring, statically typed, compiler-based language. Though it declares to be giving ease like dynamically typed interpreted code.<br />
_________________________<br />
<br />
<b>On your mark, </b><i>Get Set <b>GO</b></i><br />
<i>(getting started with the quick boost usage)</i><br />
<br />
To directly start playing with Go Lang, visit <b><a href="http://play.golang.org/">http://play.golang.org/</a>,</b><br />
where you can directly type/paste in your go-lang code in an online editor and run to get output.<br />
<br />
Just a small ++HelluvaWorld code-piece<br />
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;">package main</span> </blockquote>
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;"> import ("fmt"</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> "time"</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> "os"</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> "math" )</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;"> func main() {<br /> fmt.Println("Today is ", time.Now().Weekday())<br /> fmt.Println("env as ", os.Environ())</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"></span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> fmt.Println("A Pi on Ceil looks like ",</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> math.Ceil(math.Pi), </span><br />
<span style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> " and a Pie on Floor looks like", </span><br />
<span style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> math.Floor(math.Pi))</span><br />
<span style="font-family: 'Courier New', Courier, monospace;">}</span></blockquote>
<b>[] </b><i>Installing it for local & full-flown development practice </i><a href="http://golang.org/doc/install">http://golang.org/doc/install</a> would guide you getting 'go' working on your <a href="http://golang.org/doc/install#freebsd_linux" target="_blank">Linux, FreeBSD</a>, <a href="http://golang.org/doc/install#osx" target="_blank">OSX</a> & <a href="http://golang.org/doc/install#windows" target="_blank">Win</a> platforms.<br />
_________________________<br />
<br />
<b>Rewind before </b>the <b style="font-style: italic;">Start Line </b>and take your <b>First <i>Leap</i></b><br />
<i>(first useful step to starting use of Go Lang)</i><br />
<br />
<b>[]</b> quickie at <b>variables</b> and <b>constants, </b><i>a look at <a href="http://golang.org/doc/articles/gos_declaration_syntax.html" target="_blank">GO's declaration style</a></i><br />
<blockquote class="tr_bq">
<span style="font-family: 'Trebuchet MS', sans-serif;"><b> // </b>var used to declare variable with type at end<br />var a, b, c int<br /><b> // </b>direct initialization doesn't require providing type<br />var x, y, z = 1, true, "yes"<br /><b> // </b>constants just require a 'const' keyword<br />const newconst = 10<br />func tellvar() {<br /> a, b, c = newconst + 1, newconst + 2, newconst + 3<br /><b> // </b>inside a function, even := construct<br /><b> // </b>could be used to assign and not use 'var'<br /> clang, java, ruby := "dRitchie", "jGosling", "Matz"<br /> fmt.Println(a, b, c, x, y, z, clang, ruby, java)<br />}</span></blockquote>
now, you also know '//' is to comment as in C/C++ and more.<br />
<div>
<br />
<b>[] </b>mobilizing<b> functions</b><br />
just an emulation of 'math' libraries 'pow' method (<i><b>also </b>a look at using <b>for loop</b></i>)<br />
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;">package main</span> </blockquote>
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;">import "fmt"</span> </blockquote>
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;">func pow(x int, y int) int {<br /> a := 1<br /> for i := 0; i < y; i++ {<br /> a = a * x<br /> }<br /> return a<br />}</span> </blockquote>
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;">func main() {<br /> fmt.Println( "2 to the power of 5 is ", pow(2, 5) )<br />}</span></blockquote>
<br />
<b>[]</b> some<b> function parameters style</b>, the pow above is same as<br />
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;">func pow(x, y int) int { ... }</span></blockquote>
<br />
<b>[] </b>function<b> returning multiple values</b><br />
dream come true of how a function can return any number of values (<i><b>also</b> use of <b>if condition</b></i>)<br />
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;">func plusminus(a, b int) (int, int) {</span><span style="font-family: 'Courier New', Courier, monospace;"> </span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;">if a > b {return a+b, a-b}</span><span style="font-family: 'Courier New', Courier, monospace;"> </span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;">return a+b, b-a</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;">}</span></blockquote>
<div>
or could be like<br />
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;">func plusminus(a, b int) (plus, minus int) {</span><span style="font-family: 'Courier New', Courier, monospace;"> </span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;">plus = a + b</span><span style="font-family: 'Courier New', Courier, monospace;"> </span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;">if a > b {</span><span style="font-family: 'Courier New', Courier, monospace;"> </span></blockquote>
<span style="font-family: 'Courier New', Courier, monospace;">minus = a - b</span><span style="font-family: 'Courier New', Courier, monospace;"> </span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> } else {</span><span style="font-family: 'Courier New', Courier, monospace;"> </span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> minus = b - a</span><span style="font-family: 'Courier New', Courier, monospace;"> </span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> } </span><span style="font-family: 'Courier New', Courier, monospace;"> </span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> return</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> }</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span></div>
<div>
<div>
<div>
usage:</div>
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;">plus, minus := plusminus(1, 2)<br />fmt.Println("Plus: ", plus,<br /> "\nMinus: ", minus)</span></blockquote>
<br />
<span style="font-family: Times, 'Times New Roman', serif;"><b>[]</b><i> more to go..... in more to come</i></span><br />
<span style="font-family: Times, 'Times New Roman', serif;"><br /></span><br />
<span style="font-family: Times, 'Times New Roman', serif;">_________________________</span><br />
<br />
<b>Shops </b>to <i style="font-weight: bold;">Go</i><br />
<i>(other fine links to Go, until next part of this tutorial comes)</i><br />
<ul>
<li>Home Portal: <a href="http://golang.org/">http://golang.org/</a></li>
<li>A Tour of Go: <a href="http://tour.golang.org/#1">http://tour.golang.org/#1</a></li>
<li>Google I/O 2010 Go Lang: <a href="http://www.youtube.com/watch?v=jgVhBThJdXc&feature=relmfu" target="_blank">@youtube</a></li>
<li>Google I/O 2011 Writing WebApps in Go: <a href="http://www.youtube.com/watch?v=-i0hat7pdpk&feature=relmfu" target="_blank">@youtube</a></li>
<li>Go Hangout with Brad & Andrew: <a href="http://www.youtube.com/watch?v=9kzoGaoVkqI" target="_blank">@youtube</a></li>
<li>Mailing List: <a href="http://groups.google.com/group/golang-nuts">http://groups.google.com/group/golang-nuts</a></li>
<li>IRC Channel: <a href="irc://irc.freenode.net/#go-nuts">irc://irc.freenode.net/#go-nuts</a></li>
<li>Twitter: <a href="http://twitter.com/go_nuts">http://twitter.com/go_nuts</a></li>
<li>Blog: <a href="http://blog.golang.org/">http://blog.golang.org/</a></li>
<li>Dashboard for Go Language: <a href="http://godashboard.appspot.com/project">http://godashboard.appspot.com/project</a></li>
</ul>
_________________________</div>
</div>
</div>
</div>abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com0tag:blogger.com,1999:blog-7154053114439400556.post-663483527203019762012-05-29T15:06:00.000-07:002012-07-19T15:05:48.874-07:00Puppet ~ a beginners concept guide (Part 1)<div dir="ltr" style="text-align: left;" trbidi="on">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;">Someone asked me where to start with Puppet learning. I pointed them at <a href="http://puppetlabs.com/">PuppetLabs</a> <a href="http://docs.puppetlabs.com/">online doc</a> for <a href="http://docs.puppetlabs.com/guides/introduction.html">Puppet</a>, which is actually nice for a proper understanding. </span></span><br />
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;">But for someone trying to start with Puppet, <a href="http://docs.puppetlabs.com/learning/">that documentation is a bit long</a> to read similar to any book. </span></span><br />
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;">I searched for few blogs, but didn't found any content (<b><i>short but enough, fundamentals but usable</i></b>) that I was looking for.</span></span><br />
____________________________________________________<br />
<br />
<br />
<span class="Apple-style-span" style="font-size: large;">Puppet</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><b><span class="Apple-style-span" style="font-size: x-small;">beginners concept guide</span></b></span> (Part 1)<br />
<br />
<b>[] <i>What it is? When is it required?</i></b><i> </i><span class="Apple-style-span" style="font-size: x-small; font-style: italic;">(for all new guys, who came here while just browsing internet)</span><br />
Puppet is an OpenSource automated configuration management framework (<i>which means a tool that knows how to configure all machines to a deterministic state once you provide it the required set of manifests pulling the correct strings</i>).<br />
It's managed at enterprise level by an organization called <a href="http://puppetlabs.com/" target="_blank">PuppetLabs</a> (<a href="http://puppetlabs.com/" target="_blank">http://puppetlabs.com/</a>).<br />
<br />
<i>It is <b>required#1</b></i> when you have a hell-lot of machines required to be configured in a similar form.<br />
<i>It is <b>required#2</b></i> when you have a infrastructure requirement of dynamic scale-up and scale-down of machines with a pre-determined (or at least metadata-calculated) configuration.<br />
<i>It is <b>required#3</b> </i>to have a control over all set of configured machines, so a centralized (master-server or repo-based) change gets propagated to all automatically.<br />
<i>And more scenarios come up as you require it.</i><br />
<br />
_____________________________________<br />
<br />
<b><br class="Apple-interchange-newline" />[] <i>Quickie.</i></b><br />
Install <a href="http://www.ruby-lang.org/en/downloads/" target="_blank">Ruby</a>, <a href="http://rubygems.org/" target="_blank">Rubygems</a> on your machine where you aim to test it.<br />
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;">$ <b>gem install <a href="http://www.blogger.com/"><span id="goog_1737753961"></span>puppet<span id="goog_1737753962"></span></a> </b>--no-ri --no-rdoc</span></blockquote>
<blockquote class="tr_bq">
Download installers <a href="http://docs.puppetlabs.com/windows/installing.html" target="_blank">@Windows</a> <a href="http://downloads.puppetlabs.com/mac/" target="_blank">@MacOSX</a> ::<b>&</b>:: <i><a href="http://docs.puppetlabs.com/guides/installation.html" target="_blank">Docs to installing.</a></i></blockquote>
<br />
Checking, if it's installed properly and acting good<br />
<blockquote class="tr_bq">
Now, '<b><span style="font-family: 'Courier New', Courier, monospace;">puppet --version</span></b>' shall give you the version of installed puppet once succeed.<br />
Executing '<b><span style="font-family: 'Courier New', Courier, monospace;">facter</span></b>' and you shall get a list of System Environment related major information.</blockquote>
<br />
Have a quick puppet run, this shall create a directory '/tmp/pup' if absent. Creates a file '/tmp/pup/et' with 'look at me' as its content. <br />
<i>{In case of trying out on platforms without '/tmp' location. Like for Windows, change '/tmp' with 'C:/' or so}</i><br />
<span style="font-family: Georgia, 'Times New Roman', serif;"><br />$ <b>puppet apply -e "file</b>{'<b>/tmp/pup</b>': <br /> ensure => 'directory'} <br /> <b>file</b>{ '<b>/tmp/pup/et</b>':<br /> ensure => 'present',<br /> content => 'look at me',<br /> require => File['/tmp/pup']}<br /> <b> "</b></span><br />
_____________________________________
<br />
<br />
<br />
<b>[] <i>Dumb usage structure.</i></b><br />
Create huge manifest for your node with all resources & data mentioned in it. Then directly apply that manifest file instead of<i> '-e "abc{.....xyz}"'</i>.<br />
<br />
Say if the example above is your entire huge configuration commandment for the node. Copy all that to a file say '<b><i>mynode.pp</i></b>'.<br />
Then apply it similarly like<br />
<span style="font-family: 'Courier New', Courier, monospace;">$ <b>puppet apply mynode.pp</b></span><br />
<br />
_____________________________________
<br />
<br />
<br />
<b>[] <i>How it evolves?</i></b><br />
<br />
Now, as any application had pluggable library components to be loaded and shared as and when required. Puppet too have a concept of modules. These modules can have manifests, files-serving and more.<br />
<br />
Modules can be created in any design preference. Normally it works by having different modules per system component. To entertain different logical configuration states for any given system component (and also keeping it clean) further re-factoring can be done in the modules' manifest dividing it into different scopes.<br />
<br />
Taking <b><i>example of a module for 'apache httpd'</i></b>. For a very basic library, you might wanna structure your module like<br />
<br />
<ul style="text-align: left;">
<li>a directory base for your module: <modulepath><b>
<span style="font-weight: normal; text-align: -webkit-auto;"><</span><span style="font-weight: normal;">MODULE_PATH></span>httpd/</b></modulepath></li>
<li><b style="text-align: -webkit-auto;"><span style="font-weight: normal;">a directory in module to serve static files: <modulepath> <<span style="text-align: left;">MODULE_PATH></span>/httpd/</modulepath></span>files</b></li>
<li><span style="text-align: -webkit-auto;">static configuration file for httpd: </span><span style="text-align: -webkit-auto;"> <<span style="text-align: left;">MODULE_PATH></span>/httpd/files/<b>myhttpd.conf</b><br /><b>AccessFileName .acl</b></span></li>
<li>directory to hold your manifests in module: <modulepath> <span style="text-align: -webkit-auto;"><</span>MODULE_PATH>/httpd/<b>manifests/</b></modulepath></li>
<li>a complete solution manifest: <modulepath> <span style="text-align: -webkit-auto;"><</span>MODULE_PATH>/httpd/manifests/<b>init.pp<br />class httpd{<br /> include httpd::install<br /> include httpd::config<br /> include httpd::service<br />}</b></modulepath></li>
<li>a manifest just installing httpd: <span style="text-align: -webkit-auto;"><modulepath> <<span style="text-align: left;">MODULE_PATH></span>/httpd/manifests/<b>install.pp<br />class httpd::install {</b></modulepath></span><b style="text-align: -webkit-auto;"> package {'httpd': ensure => 'installed'}<br />}</b></li>
<li><span style="text-align: -webkit-auto;">a manifest just configuring httpd: </span> <span style="text-align: -webkit-auto;"><modulepath>
<<span style="text-align: left;">MODULE_PATH></span>/httpd/manifests/<b>config.pp</b></modulepath></span>
<span style="text-align: -webkit-auto;"><b><br />class httpd::config{</b></span><b style="text-align: -webkit-auto;"> file {'/etc/httpd/conf.d/httpd.conf':<br /> ensure => 'present',<br /> source => 'puppet:///modules/httpd/myhttpd.conf'<br /> }<br />}</b></li>
<li><span style="text-align: -webkit-auto;">a manifest just handling httpd service: <modulepath> <<span style="text-align: left;">MODULE_PATH></span>/httpd/manifests/<b>service.pp<br />class httpd::service{<br /> service{'httpd': ensure => 'running' }<br />}</b></modulepath></span></li>
</ul>
<br />
<i>Now, using it</i><br />
<br />
$ <b>puppet apply --modulepath=<MODULE_PATH> -e "include httpd"</b><br />
would install, custom-configure and start the httpd service.<br />
<br />
<br />
$ <b>puppet apply --modulepath=<MODULE_PATH> -e "include httpd::install"</b><br />
would just install the httpd service.<br />
<br />
<br />
<br />
________________________________________________________________<br />
<h4 style="text-align: left;">
<a href="http://justfewtuts.blogspot.com/2012/07/puppet-beginners-concept-guide-part-2.html" target="_blank"><b>Part2: Road to Modules</b></a></h4>
</div>abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com9tag:blogger.com,1999:blog-7154053114439400556.post-66749096767291463882012-03-24T17:47:00.000-07:002012-03-24T22:28:31.906-07:00xml-motor ~ what it is; how & why should you use it<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: right;">
<i><b>xml-motor ~ what it is; why & how you should use it</b></i></div>
<div>
<br />
Download this article as pdf on what,why,how<br />
<a href="http://speakerdeck.com/u/abhishekkr/p/xml-motor-whatwhyhow-this-xml-parsing-rubygem#">http://speakerdeck.com/u/abhishekkr/p/xml-motor-whatwhyhow-this-xml-parsing-rubygem#</a><br />
<br />
<script src="http://speakerdeck.com/embed/4f6e6818933f08001f00b579.js">
</script>
<br />
or read it all here.....<br />
<br /></div>
Late 2011, I started with a new rubygem project for parsing xml, html content.<br />
<b><i>@Rubygems:</i></b> <a href="http://rubygems.org/gems/xml-motor">http://rubygems.org/gems/xml-motor</a><br />
<b><i>@GitHub :</i></b> <a href="https://github.com/abhishekkr/rubygem_xml_motor">https://github.com/abhishekkr/rubygem_xml_motor</a><br />
<br />
Just created it to test out my work at compact, quick & easy xml-parsing algorithm... can see that<br />
<b><i>@Slideshare:</i></b> <a href="http://www.slideshare.net/AbhishekKr/xmlmotor">http://www.slideshare.net/AbhishekKr/xmlmotor</a><br />
<br />
So, currently this is a <b><i>non-native</i></b>, completely <b><i>independent less-than-250 ruby-LOC</i></b> available as a simple rubygem to be require-d and <b><i>use in an easy freehand notation and match with any node attributes</i></b>.<br />
<br />
<b>Current Features:</b><br />
<ul style="text-align: left;">
<li>Has a single method access to parse require xml nodes from content or file. Use it only if you are gonna parse that xml-content once. For using same xml-content more than once, follow the 3-way step mentioned in examples.</li>
<li>It doesn't depend on presence of any other system library, purely non-native.</li>
<li>It parses broken or corrupted xml/html content correctly, just for the content it have.</li>
<li>Can parse results on looking for node-names, attributes of node or both.</li>
<li>Uses free-freehand notation to retrieve desired xml nodes<br />if your xml looks like,<br /><i><span class="Apple-style-span" style="font-family: Times, 'Times New Roman', serif;">'<library>...<br /> <book> <title>ABC</title> <author>CBA</author> </book>...<br /> <book><br /> <title>XYZ</title><br /> <authors> <author>XY</author><author>YZ</author> </authors></book>... <br /></library>'</span></i><br />and you look for 'book.author',<br />then, you'll get back <i>['CBA', 'XY', 'YZ']</i>;<br />what that means is the child-node could be at any depth in the parent-node.</li>
<li>Default return mode is without the tags, there is a switch to get the nodes.<br />as you'd have seen in above example:<br />'<i>CBA</i>' gets sent by default, not '<i><author>CBA</author></i>'</li>
<li>To filter your nodes on the basis of attributes, single or multiple attributes can be provided.</li>
<li>These attribute searches can be combined up with freehand node name searches.</li>
<li><b><i>Readme (a bit weird):</i></b> <a href="https://raw.github.com/abhishekkr/rubygem_xml_motor/master/README">https://raw.github.com/abhishekkr/rubygem_xml_motor/master/README</a></li>
</ul>
<div>
<b><br /></b></div>
<div>
<b><br /></b></div>
<div>
<b>Features To Come:</b><br />
<ul style="text-align: left;">
<li>Work on making it more performance efficient.</li>
<li>Limit over result-nodes retrieved from start/end of matching nodes.</li>
<li>Multi-node attribute-based filter for a hierarchical node search.</li>
<li>Add dev-knows CSS Selector, it's already present using attribute based search... just need to add a mapping method.</li>
</ul>
<div>
<br /></div>
<div>
<br /></div>
<b>EXAMPLES of usage:</b><br />
<b><i>example code to try: </i></b><a href="https://github.com/abhishekkr/axml-motor/tree/master/ruby/examples">https://github.com/abhishekkr/axml-motor/tree/master/ruby/examples</a></div>
<div>
<ul style="text-align: left;">
<li>say, you have an xml file <b><i>'dummy.xml'</i></b>, with data as<br /><i><dummy><br /> <ummy> <mmy class="sys">non-native</mmy> </ummy><br /> <ummy><br /> <mmy class="sys"> <my class="sys" id="mem">compact</my> </mmy><br /> </ummy><br /> <mmy type="user"> <my class="usage">easy</my> </mmy></dummy></i></li>
<li>its available at rubygems.org, install it as<br /> $ <i>gem install xml-motor</i></li>
<li>include it in your ruby code,<br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> #!/usr/bin/env ruby<br /> require 'xml-motor'</span></li>
<li>get the XML Filename and/or XML data available<br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> fyl = </span><span class="Apple-style-span" style="font-family: Times, 'Times New Roman', serif;">File.join(File.expand_path(File.dirname __FILE__),</span><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">'dummy.xml'</span><span class="Apple-style-span" style="font-family: Times, 'Times New Roman', serif;">)</span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> xml = File.open(fyl,'r'){|fr| fr.read }</span></li>
<li>One-time XML-Parsing directly from file<br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> XMLMotor.get_node_from_file(fyl, 'ummy.mmy', 'class="sys"')</span><br /><span class="Apple-style-span" style="font-family: Times, 'Times New Roman', serif;"><i> Result: </i></span><span class="Apple-style-span" style="font-family: Times, 'Times New Roman', serif;">["non-native", "\n <my class="\"sys\"" id="\"mem\"">compact</my>\n "]</span></li>
<li>One-time XML-Parsing directly from content<br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> XMLMotor.get_node_from_content xml, 'dummy.my', 'class="usage"'</span><br /><span class="Apple-style-span" style="font-family: Times, 'Times New Roman', serif;"><i> Result:</i> ["easy"]</span><br /> </li>
<li>Way to go for XML-Parsing for xml node searches<br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> xsplit = XMLMotor.splitter xml<br /> xtags = XMLMotor.indexify xsplit</span><br /><i><br /> [] just normal node name based freehand notation to search:</i><br /> <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">XMLMotor.xmldata xsplit, xtags, 'dummy.my'</span><br /><span class="Apple-style-span" style="font-family: Times, 'Times New Roman', serif;"><i> Result:</i> ["compact", "easy"]</span><br /><i> [] searching for values of required nodes filtered by attribute:</i><br /> <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">XMLMotor.xmldata xsplit, xtags, nil, 'class="usage"'</span><br /><span class="Apple-style-span" style="font-family: Times, 'Times New Roman', serif;"><i> Result:</i> ["easy"]</span><br /><i><br /> [] searching for values of required nodes filtered by freehand tag-name notation & attribute:</i><br /> <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">XMLMotor.xmldata xsplit, xtags, 'dummy.my', 'class="usage"'</span><br /><span class="Apple-style-span" style="font-family: Times, 'Times New Roman', serif;"><i> Result:</i> ["easy"]</span><br /><i><br /> [] searching for values of required nodes filtered by freehand tag-name notation & multiple attributes:</i><br /> <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">XMLMotor.xmldata xsplit, xtags, 'dummy.my', ['class="sys"', 'id="mem"']</span><br /><span class="Apple-style-span" style="font-family: Times, 'Times New Roman', serif;"><i> Result:</i> ["compact"]</span></li>
</ul>
</div>
</div>abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com0tag:blogger.com,1999:blog-7154053114439400556.post-73175615972860534632012-03-05T11:05:00.004-08:002012-03-05T11:08:10.637-08:00messQ ~ just a fun little project providing socket-based Queue service<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-NvYG_y9DkIKQuHLCh-xIryeZkP0tr_K-FfDOuKZ3VzBpPxnoUKj-kZBXiKayt6YCOH7o1bBy_yDFD6pBhxZYldNB0I_nZuE_w9cyCi8XBgBZd3KKTQM9kDv7LwRn5voIiXO0S0I22Gs/s1600/messQ_small.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="150" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-NvYG_y9DkIKQuHLCh-xIryeZkP0tr_K-FfDOuKZ3VzBpPxnoUKj-kZBXiKayt6YCOH7o1bBy_yDFD6pBhxZYldNB0I_nZuE_w9cyCi8XBgBZd3KKTQM9kDv7LwRn5voIiXO0S0I22Gs/s200/messQ_small.jpg" width="200" /></a><b><span style="font-family: Verdana, sans-serif; font-size: large;">messQ </span></b>is a small project started to implement and improve in the areas of message queue mechanisms.<br />
<br />
<b><i><span style="font-family: 'Trebuchet MS', sans-serif;">What it does currently? </span></i></b>Just a Network Service to be connect and enqueue/dequeue messages.<br />
<br />
<b><i><span style="font-family: 'Trebuchet MS', sans-serif;">What it requires? </span></i></b>Ruby, terminal and your fingers :)<br />
<br />
<i><b>Git it:</b></i> $ <b>git clone git://github.com/abhishekkr/messQ.git</b><br />
<b><i>Download: </i></b><a href="https://github.com/abhishekkr/messQ/tarball/master">https://github.com/abhishekkr/messQ/tarball/master</a><br />
<br />
<i><b>Start messQ server:</b></i> $<i> </i><b>ruby messQ.rb</b><br />
This starts a message queue server at <i><span style="font-family: 'Courier New', Courier, monospace;">port 8888</span></i>.<br />
<br />
<b><i>Enque new message:</i></b><br />
Open a connection at <b><span style="font-family: 'Courier New', Courier, monospace;">port 8888</span></b>, then say "<b><span style="font-family: 'Courier New', Courier, monospace;">enq MESSAGE_TO_BE_QUEUED</span></b>".<br />
<br />
<b><i>Deque oldest message:</i></b><br />
Open a connection at <b><span style="font-family: 'Courier New', Courier, monospace;">port 8888</span></b>, then say "<b><span style="font-family: 'Courier New', Courier, monospace;">deq</span></b>". It returns the dequed message.<br />
<br />
<br />
<pre style="background-color: white; border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #333333; font-family: 'Bitstream Vera Sans Mono', Courier, monospace; font: inherit; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: -webkit-auto; white-space: pre-wrap;">+++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++++++++++++++
<span style="font-family: 'Courier New', Courier, monospace;"> _ _ __ ___ ___ _____
||\/|| //_ //_ //_ _ // // messQ v0.0.1beta
|| || \\_ __// __// //__//\\_ simplistic socket message Q</span>
+++++++++++++++++++++++++++++++++++++++++++++++++++++++</pre>
<pre style="background-color: white; border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #333333; font-family: 'Bitstream Vera Sans Mono', Courier, monospace; font: inherit; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: -webkit-auto; white-space: pre-wrap;"></pre>
<pre style="background-color: white; border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #333333; font-family: 'Bitstream Vera Sans Mono', Courier, monospace; font: inherit; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: -webkit-auto; white-space: pre-wrap;"></pre>
</div>abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com0tag:blogger.com,1999:blog-7154053114439400556.post-42602929431811033012011-12-14T19:03:00.000-08:002011-12-18T08:45:03.818-08:00Basics of Powershell ~ empowering Windows Config/Admin Automation<div dir="ltr" style="text-align: left;" trbidi="on">
<span class="Apple-style-span" style="font-family: Georgia, 'Times New Roman', serif;"><i>Basics of </i><b>MS Windows Powershell</b></span><br />
<span class="Apple-style-span" style="font-family: Georgia, 'Times New Roman', serif;"><b><br /></b></span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjX-FvTa-XLqxFuLXxkgCf-7huTzaII6atxhqvDUl1EOGJOj21jNsdHLbo0GQltR-QVOXbgCf8QEHJDi-ByTsIUOD2dD4R31IS1XGkSMYQpGoBTaIzIV8RPAvZ3UACHCROtcoIsqj-xXxA/s1600/powershell_joke.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="244" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjX-FvTa-XLqxFuLXxkgCf-7huTzaII6atxhqvDUl1EOGJOj21jNsdHLbo0GQltR-QVOXbgCf8QEHJDi-ByTsIUOD2dD4R31IS1XGkSMYQpGoBTaIzIV8RPAvZ3UACHCROtcoIsqj-xXxA/s320/powershell_joke.jpg" width="320" /></a></div>
<span class="Apple-style-span" style="font-family: Georgia, 'Times New Roman', serif;"><b><i>Introduction</i></b></span><br />
A dotNet framework based scripting language to automate the configuration/administration of Microsoft Windows machine.<br />
Powershell is<b><i> loaded with several cmdlets</i></b> (special command-lets) acting as a built-in shell utilities to perform different tasks on Windows machine for performing administrative tasks.<br />
<br />
<br />
<span class="Apple-style-span" style="font-family: Georgia, 'Times New Roman', serif;"><b><i>Getting Started</i></b></span><br />
Powershell's cmdlets act upon and return objects as a result of any action taken.<br />
These can be used in combination with traditional Windows services like Registry, Net and more.<br />
<br />
To try your hands over powershell, access it at <span class="Apple-style-span" style="font-family: Times, 'Times New Roman', serif;"><b>'Start Menu' > 'Accessories' > 'Windows Powershell'</b></span>;<br />
there you'll get mainly a shell from<span class="Apple-style-span" style="font-family: Times, 'Times New Roman', serif;"><b> 'Powershell'</b></span>, and an interactive IDE-like shell <span class="Apple-style-span" style="font-family: Times, 'Times New Roman', serif;"><b>'Powershell ISE'</b></span>.<br />
<br />
Using powershell, cmdlets are the main power-source of this Powershell which are discussed briefly below.....<br />
<br />
<i><b>to get a quick </b>/Hello World/<b> feel of Powershell, you could try on few next steps</b></i><br />
* emulating a dos command<i> 'echo'</i> used to print message at console,<i> </i><br />
<i> command: </i><br />
<i> <span class="Apple-style-span" style="font-family: 'Trebuchet MS', sans-serif;"> </span></i><b><span class="Apple-style-span" style="font-family: 'Trebuchet MS', sans-serif;">Write-Host "[get-help write-host] would tell about cmdlet write-host"</span></b><br />
* emulating a dos command<i> 'cd' </i>used to change current directory,<i> </i><br />
<i> command: </i><span class="Apple-style-span" style="font-weight: bold;"><span class="Apple-style-span" style="font-family: 'Trebuchet MS', sans-serif;">Set-Location C:\Temp</span></span><br />
* emulating a dos command<i> 'mkdir' </i>used to create directory,<i> </i><br />
<i> command: </i><span class="Apple-style-span" style="font-weight: bold;"><span class="Apple-style-span" style="font-family: 'Trebuchet MS', sans-serif;">New-Item -name "ztemp" -type directory -Force</span></span><br />
<div>
<div>
* emulating a dos command<i> 'dir'</i> used to list items in current directory,<i> </i><br />
<i> command: </i><span class="Apple-style-span" style="font-weight: bold;"><span class="Apple-style-span" style="font-family: 'Trebuchet MS', sans-serif;">Get-ChildItem -path C:\Temp</span></span></div>
<div>
* even the windows '<b>dir</b>' command can be executed and played with inside powershell.<br />
<i>.....could get a list of cmdlets available in Powershell2.0 ~ <a href="http://ss64.com/ps/">http://ss64.com/ps/</a></i><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-k6CUEAaehgB7z8gCcLnOJibDdsTNBzeC9tTYTSXoeh3PdmCVu9FycexKd7UKBSkafKmEd9kEBQEF5rXl2w1hZZgLkj-tExE2xCXVbhWN9kUeqM5YIq3BbeYbkHFUW3R63waZOrRm-4Y/s1600/powershell.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-k6CUEAaehgB7z8gCcLnOJibDdsTNBzeC9tTYTSXoeh3PdmCVu9FycexKd7UKBSkafKmEd9kEBQEF5rXl2w1hZZgLkj-tExE2xCXVbhWN9kUeqM5YIq3BbeYbkHFUW3R63waZOrRm-4Y/s1600/powershell.jpg" /></a></div>
<br />
<div style="text-align: left;">
<b><span class="Apple-style-span" style="font-family: 'Trebuchet MS', sans-serif;">[] cmdlets</span></b></div>
Command-lets have a specific name format of<b> /'verb'-'noun'/ </b>such as 'Get-ChildItem', 'Get-Help', etc.<br />
To know more about any <b>'cmdlet'</b> (like using<i> man</i> in linux shell), use<br />
<i>command:</i> <b><span class="Apple-style-span" style="font-family: 'Trebuchet MS', sans-serif;">Get-Help <_cmdlet_> -detailed</span></b><br />
<br />
<b><span class="Apple-style-span" style="font-family: 'Trebuchet MS', sans-serif;">[] </span><i>A </i><span class="Apple-style-span" style="font-family: 'Trebuchet MS', sans-serif;">Glimpse of<i> system level </i>SuperUser stuff</span><i>,</i></b><i> as possible on a linux shell</i><br />
<b>[+]</b> to get a <i>listing of all System Services</i>,<br />
<i>command:</i> <b><span class="Apple-style-span" style="font-family: 'Trebuchet MS', sans-serif;">Get-Service</span></b><br />
<b>[+]</b> to just get a listing of <i>'Spooler'</i> named system service, telling its 'name','status' and 'display name'<br />
<i>command:</i> <span class="Apple-style-span" style="font-family: 'Trebuchet MS', sans-serif;"> <b>Get-Service | Where {$_.name -eq 'Spooler'}</b></span><br />
<b>[+]</b> to get current state of <i>'Spooler'</i> named system service, whether its running/stopped/paused<br />
<i>command:</i> <b><span class="Apple-style-span" style="font-family: 'Trebuchet MS', sans-serif;">Get-Service | Where {$_.name -eq 'Spooler'} | %{ $_.status }</span></b><br />
<b> [+] </b>to have a Powershell script, check for a system service..... start it if stopped<br />
<blockquote class="tr_bq" style="background-color: black; color: lime;">
<span class="Apple-style-span" style="font-family: 'Trebuchet MS', sans-serif; font-size: x-small;"> </span><span class="Apple-style-span" style="font-family: 'Trebuchet MS', sans-serif;">$svc_name = "<i>aspnet_state</i>"<br /><b> $svc_status = Get-Service | Where {$_.name -eq $svc_name} | %{ $_.status }</b><br /> if (-not $svc_status) {<br /><b> Write-Host "Error: $svc_name not found"</b><br /> }<br /><b> </b> elseif ($svc_status -eq "running"){<br /><b> Write-Host "status ok $svc_name"</b><br /><b> </b> } else {<br /><b> Start-Service $svc_name</b><br /> }</span></blockquote>
<b> [+] N</b>ow you can <i>save the script</i> above as any file say <i>'start_aspnet.ps1'</i>, but to execute it as an external script you would need the local system's Execution Policies to be unrestricted for the script.<br />
<b><i>It's not advisable to have it un-restricted all the times, so you could pass on the specific modes along-with the script you desire to be run in un-restricted mode.</i></b> As below.....<br />
<i>cmd_prompt:></i> <b><span class="Apple-style-span" style="font-family: 'Trebuchet MS', sans-serif;">powershell -executionpolicy unrestricted -file ".\start_aspnet.ps1"</span></b><br />
<br />
<b><br /></b><br />
<b><i>helpful links:</i></b><br />
<a href="http://msdn.microsoft.com/en-us/library/windows/desktop/aa973757(v=vs.85).aspx">http://msdn.microsoft.com/en-us/library/windows/desktop/aa973757(v=vs.85).aspx</a><br />
<a href="http://blogs.msdn.com/b/powershell/">http://blogs.msdn.com/b/powershell/</a><br />
<a href="http://thepowershellguy.com/blogs/posh/default.aspx">http://thepowershellguy.com/blogs/posh/default.aspx</a><br />
<a href="http://powershell.com/cs/">http://powershell.com/cs/</a><br />
<a href="http://stackoverflow.com/questions/496234/what-tutorial-do-you-recommend-for-learning-powershell">http://stackoverflow.com/questions/496234/what-tutorial-do-you-recommend-for-learning-powershell</a></div>
</div>
</div>abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com0tag:blogger.com,1999:blog-7154053114439400556.post-85407668568224061502011-11-12T06:30:00.001-08:002011-11-12T07:55:27.084-08:00XML-Motor v0.0.3 ~ mine XML once..... multiple search<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-size: x-small;">released </span>v0.0.3<span style="font-size: x-small;"> of </span>xml-motor <span style="font-size: x-small;">with following update</span>: <b><span style="font-family: Georgia,"Times New Roman",serif; font-size: small;"><a href="http://rubygems.org/gems/xml-motor">http://rubygems.org/gems/xml-motor</a></span></b> <br />
<br />
<b>Earlier</b>:<br />
* it just supported one search per every XML data processing<br />
<b>Now</b>:<br />
* could retrieve processed XML 'Information' once, and search again & again with minimized cost<br />
<br />
<b>Now, along-with..... </b><br />
* a single method-call search for Tag (w/ or w/o Attribute) in provided XML String or File<br />
<b>You also have the capability to.....</b><br />
* fetch the two (<i><span style="font-size: x-small;">or one, depends how you wanna roll</span></i>) required set of Information mined by providing it XML String<br />
* then provide that information everytime to search anything, and hence making it way quicker for multiple searches<br />
<br />
<br />
<div style="font-family: "Trebuchet MS",sans-serif;">
<i><b><span style="font-size: large;">So, the improved way for multiple searches:</span></b></i></div>
<br />
First, update your 'xml-motor' gem using<br />
$ <b><span style="font-family: "Courier New",Courier,monospace;">gem install xml-motor</span></b><br />
<br />
Now, the way to use it<br />
<blockquote class="tr_bq" style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: small;"><b>#!/usr/bin/env ruby </b></span><br />
<span style="font-size: small;"><b>require 'xml-motor'</b></span><br />
<br />
<span style="font-size: small;"><b>fyl = File.open "myXMLfile.xml"</b></span><br />
<span style="font-size: small;"><b>str = fyl.read</b></span><br />
<span style="font-size: small;"><b>xNodes = XMLMotorEngine._splitter_ str</b></span><br />
<span style="font-size: small;"><b>xTags = XMLMotorEngine._indexify_ xNodes</b></span><br />
<span style="font-size: small;"><b>result1 = XMLMotorEngine.pre_processed_content(<br /> xNodes, xTags, "h1")</b></span><br />
<span style="font-size: small;"><b>result2 = XMLMotorEngine.pre_processed_content(<br /> xNodes, xTags, "div", "class='wanted'")</b></span></blockquote>
<br /></div>abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com0tag:blogger.com,1999:blog-7154053114439400556.post-69796156158542899992011-11-09T13:30:00.000-08:002011-11-09T13:41:00.652-08:00[new portable algo] Easy XML Parsing using XML-Motor (currently just in Ruby)<div dir="ltr" style="text-align: left;" trbidi="on">
Easy <b>XML Parsing </b>using<b> XML-Motor </b><br />
<span style="font-size: x-small;">(currently just implemented in Ruby as a Gem)</span><br />
<span style="font-size: x-small;"><span style="font-size: small;"><b style="font-family: Georgia,"Times New Roman",serif;">Using a new, compact algorithm of XML Partsing</b></span> </span><br />
<br />
===================================================<br />
v 0.0.2<br />
@<b>GitHub</b>: <a href="https://github.com/abhishekkr/rubygem_xml_motor">https://github.com/abhishekkr/rubygem_xml_motor</a><br />
<br />
@<b>RubyGems</b>: <a href="http://rubygems.org/gems/xml-motor">http://rubygems.org/gems/xml-motor</a><br />
===================================================<br />
<br />
A new short XML Parsing Algorithm implemented in LessThan 200 Ruby lines.<br />
An easy-to-use XML Parser without any Native Dependencies.<br />
<br />
<div style="text-align: center;">
<b><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">[How To Use]</span></span></b></div>
<br />
<b><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Installing and Loading:</span></span></b><br />
+ $ <b>gem install xml-motor</b><br />
+ in your ruby code: <b>require 'xml-motor'</b><br />
<br />
<span style="font-size: large;"><b style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Usage:</b></span><br />
+ To find values of an xml node from an xml file<br />
<b><span style="font-family: Times,"Times New Roman",serif;">XMLMotor.get_node_from_file [file_name_with_path], [tag_name]<file_with_path><node></node></file_with_path></span></b><br />
+ To find values of an xml node from an xml string<br />
<b><span style="font-family: Times,"Times New Roman",serif;">XMLMotor.get_node_from_content [xml_data], [tag_name]<xml_string><node></node></xml_string></span></b><br />
+ To find values of an xml node with a tag_name having required attribute<br />
<b><span style="font-family: Times,"Times New Roman",serif;">XMLMotor.get_node_from_content</span></b><b><span style="font-family: Times,"Times New Roman",serif;"> [xml_data], [tag_name], [attr_key=attr_value]</span></b><br />
<br />
<span style="font-size: large;"><b><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Example Calls As Code:</span></b></span><br />
+ <span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">XMLMotor</span>.<span style="font-size: x-small;">get_node_from_content</span> <span style="font-size: x-small;">"</span></b></span><b><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><</span></b></span></b><span style="font-size: x-small;"><b style="font-family: Arial,Helvetica,sans-serif;">A i='1'</b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">>a</span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><</span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><b><c>A</c></b></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><b><c><B</c></b></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><b><c>b<C</c></b></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><b><c>c<C</c></b></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">><A</span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><b><c></c>ba<A</b></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">><B</span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">"</span>, "A"</b></span><br />
RETURNS: ["a", "ba"]<br />
+ <b><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">XMLMotor</span>.<span style="font-size: x-small;">get_node_from_content</span> <span style="font-size: x-small;">"</span></span></b><b><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><</span></b></span></b><span style="font-size: x-small;"><b style="font-family: Arial,Helvetica,sans-serif;">A i='1'</b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">>a</span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><</span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><b>A</b></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><b><B</b></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><b>b<C</b></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><b>c<C</b></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">><A</span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><b>ba<A</b></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">><B</span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">></span></b></span><span style="font-size: x-small;"><b style="font-family: Arial,Helvetica,sans-serif;"><b></b></b></span><b><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">"</span>, "B.A"</span></b><br />
RETURNS: ["ba"]<br />
+ <b><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">XMLMotor</span>.<span style="font-size: x-small;">get_node_from_content "</span></span></b><b><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><</span></b></span></b><span style="font-size: x-small;"><b style="font-family: Arial,Helvetica,sans-serif;">A i='1'</b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">>a</span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><</span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><b>A</b></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><b><B</b></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><b>b<C</b></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><b>c<C</b></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">><A</span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;"><b>ba<A</b></span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">><B</span></b></span><span style="font-size: small;"><b style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">></span></b></span><b><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: x-small;">"</span>, "A", <span style="font-size: x-small;">"i='1'"</span></span></b><br />
RETURNS: ["a"]<br />
<br />
===================================================<br />
<br />
<div style="text-align: center;">
<b><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">[Example Code]</span></span></b></div>
<br />
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<pre><span style="font-size: large;"><b style="font-family: "Courier New",Courier,monospace;">require 'xml-motor'
puts XMLMotor.get_node_from_file "eloqjs.htm", "h1"
fyl = File.open("elogjs.htm")
XMLData = fyl.read
puts XMLMotor.get_node_from_content XMLData, "div.span"
puts XMLMotor.get_node_from_content XMLData, "div"</b></span></pre>
</blockquote>
</blockquote>
<br />
===================================================<br />
<br />
<div id="__ss_10062216" style="width: 477px;">
<b style="display: block; margin: 12px 0pt 4px;"><a href="http://www.slideshare.net/AbhishekKr/xmlmotor" target="_blank" title="XML-Motor">XML-Motor</a></b> <iframe frameborder="0" height="510" marginheight="0" marginwidth="0" scrolling="no" src="http://www.slideshare.net/slideshow/embed_code/10062216" width="477"></iframe> <br />
<div style="padding: 5px 0pt 12px;">
View more <a href="http://www.slideshare.net/" target="_blank">documents</a> from <a href="http://www.slideshare.net/AbhishekKr" target="_blank">Abhishek Kumar</a> </div>
</div>
</div>abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com0tag:blogger.com,1999:blog-7154053114439400556.post-75161671855904145742011-10-20T09:23:00.000-07:002011-10-20T09:24:43.600-07:00http_voodoo_mongo ~ remote control your MongoDB over HTTP<div dir="ltr" style="text-align: left;" trbidi="on">
@github: <a href="https://github.com/abhishekkr/http_voodoo_mongo"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">https://github.com/abhishekkr/http_voodoo_mongo</span></a><br />
<div class="separator" style="clear: both;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyJCLLah-5T2z4411PaY4ldpw04Eynn863KcQBzgQQo21CPWQrPXxmLZIIg92-OTCMIV2CtpN4EBI7qDUdgLaGvOpn0jtSqFwMFHE3Zv_s0-_OH33ZrErDfuZq1qDsmPgL_n7pQW-BsIk/s1600/http_voodoo_mongodb.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyJCLLah-5T2z4411PaY4ldpw04Eynn863KcQBzgQQo21CPWQrPXxmLZIIg92-OTCMIV2CtpN4EBI7qDUdgLaGvOpn0jtSqFwMFHE3Zv_s0-_OH33ZrErDfuZq1qDsmPgL_n7pQW-BsIk/s200/http_voodoo_mongodb.jpg" width="150" /></a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
project name: <span class="Apple-style-span" style="font-family: Verdana, sans-serif;">http_voodoo_mongo</span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: Times, 'Times New Roman', serif;">platform:</span><span class="Apple-style-span" style="font-family: Verdana, sans-serif;"> linux</span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: Times, 'Times New Roman', serif;">requirement:</span><span class="Apple-style-span" style="font-family: Verdana, sans-serif;"> ruby</span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: Times, 'Times New Roman', serif;">usage:</span><span class="Apple-style-span" style="font-family: Verdana, sans-serif;"> it's a remote controller for your local/remote MongoDB instances working over an HTTP mocking service</span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: Verdana, sans-serif;">What it consists of?</span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: Verdana, sans-serif;">[] a rakefile 'Rakefile' allowing you to auto install and start 'http_voodoo_mongo' as a system service using "rake http_mongo:install" task.</span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: Verdana, sans-serif;">[] a ruby script 'mongo_voodoo.rb' which mocks HTTP Server and allows you control MongoDB using HTTP Requests.</span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: Verdana, sans-serif;">[] a system service script 'mongo_voodoo' which enables you to start||stop||status||restart the Ruby Script as a System Service</span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><b>*currently I've just added "mongodump" task to it (with & without authorization detail), will slowly add more..... it's all opensource, even you can update if you want</b></span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: left;">
<span class="Apple-style-span" style="font-family: Verdana, sans-serif;"><span class="Apple-style-span" style="font-family: 'Times New Roman';">======================================================</span></span></div>
a REST API for Remote Controlling MongoDB Server over HTTP using the same 'authentication' as of MongoDB Server<br />
<div>
<br /></div>
<div>
[] need to access the service at HTTP_URL => http://[MongoDB_IP]:30303/</div>
<div>
<br /></div>
<div>
[] Optional GET Request Parameters [name] the username permitted access [pass] the password linked to username [port] the port at which MongoDB Server is running</div>
<div>
<br /></div>
<div>
::MongoDB Database Backup </div>
<div>
http://[MongoDB_IP]:30303/[db_name]?action=dump&dir=[dir]&more_switch=[more_switch]&userniame=[username]&password=[password]&port=[port] </div>
<div>
<br /></div>
<div>
[] GET Parameters [dir] the directory to take in backup using mongodump [more_switch] any options you wanna add to mongodump command </div>
======================================================</div>abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com1tag:blogger.com,1999:blog-7154053114439400556.post-81256631606146649172011-08-09T07:49:00.000-07:002011-08-09T07:49:44.132-07:00Make *nix automatic re-spawn a process/service if stopped<div dir="ltr" style="text-align: left;" trbidi="on"><br />
To make your *nix system automatic re-spawn a process/service if stopped ~<br />
<br />
Write a shell script to check for presence of a service and start it if it is not running.<br />
Then attach that shell-script to a cronjob running at frequent intervals.<br />
<br />
Example Script to monitor Apache HTTPD web server and start it if is not started ~<br />
<blockquote><b><span style="font-family: "Courier New",Courier,monospace;">#!/bin/sh</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">pids_=`ps ax | grep /usr/sbin/httpd | grep -v grep | wc -l `</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">if [ $pids_ -ne 0 ];</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">then</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> echo "Apache is running"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">else</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> echo "Apache was stopped. Starting..."</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> apache_start_=`/etc/init.d/httpd start`</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> if [ $? -eq 0 ];</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> then</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> echo "Apache Started Successfully"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> fi</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">fi</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#############################</span></b></blockquote> this simply checks for HTTPD process to be present and re-spawn the service in absence of even a single process thread.<br />
<br />
There is a wonderful Ruby utility "<b><a href="http://www.blogger.com/">bluepill</a></b>" to achieve the same solution in a more advanced and comfortable manner. Cover it sooooon.....</div>abionichttp://www.blogger.com/profile/06276198262605731980noreply@blogger.com0