lenz: November 2007 Archives

i posted about the HA daemon earlier and wrote one now. this is how it works and what it does ... it is pretty basic but it does the job and if i have some time i'll grow it to a nice one.

the basic architecture

#cat checks.conf
[local]
check_db_host1
chech_db_host2

this defines a scope (local) and checks that should be run in this scope. currently it supports only local checks, remote checks could be implemented via ssh or via a daemon running those hosts.

#cat actions.conf
[slave_to_master]
stopservice1
unplumb
plumb %MASTER%
startservice1

this defines state changes. if the daemon sees a change for the host it monitors and the status has been "slave" before and now is "master" then these commands run in a fork. there are variables in the config file to indicate arguments like %MASTER% and %SLAVE% that get expanded to the according hostnames during runtime.

the daemon is based on the Net::Server framework that is the best framework for perl based daemons i ever worked with and that has proven to be reliable in high traffic environments during the last years already. i decided to maintain a internal basic hash table in a file based database to synchronize forks, this could move to a shared memory based solution but performance is not so much an issue in this setup so i decided that a file is good enough.

the checks are simple perl scripts that get called from a fork of the main daemon via "qx{}" and report the new status back to the daemon via a UNIX socket in for of a serialized hash.
the action scripts are simple perl scripts that, unlike the checks, are processed serial in the order defined in the config file. these shell scripts start/stop other daemons (un)/plumb network interfaces and so on.

the main goal is reached, a self contained daemon that can be copied to any location and run with near to no setup. all that has to be done is defining simple checks and configuring the state changes ... pretty straight forward.

i have a project where veritas manages various fail over szenarios with lots of middelware involved and all the corporate nightmare you want to have. the problem now is that they want to manage a fail over between two database servers and the connected middleware without veritas ... and so they asked me to write "something".

having thought about it for one day now i looks like i could write a quite general soloution that could be deployed in various situations where you have a master and a hot standby slave. so, lets write a HA daemon.

what does a HA daemon?

the main goals of a HA daemon is to monitor states and act according to state changes. that's all the magic. so how can we do that? my current idea is to implement an internal state hash that learns the environment and reads a set of rules. every now and then some checks get forked and the checks write results back to the main daemon. if the daemon encounters state changes it tries to trigger actions that were defined for those state changes. and starts monitoring the environment again.

interesting things...

well, i have never written a language parser before and that is something i really wanted to do some when so, yes, this is the moment. as the rules have to be defined somehow we need a parser that reads the rules and interprets them and moves them in any kind of internal data structure that can be used by the daemon. thins is an interesting one.
forking off checks after a certain time that report back to a main daemon. this is a cron like environment where you can define tests that run on defined times.
completely self contained ... this is a hard one. they don't want to install tons of CPAN modules so i have to write everything from scratch.

where could it go?

there is currently the plan to write the daemon and a basic CLI for it. there could, however, some when be a web interface for it, there could be a general thing that does all kinds of fail overs or simply actions ... probably i can even convince them to make an opensource project of it. lets see.

well, the we based social networking became an essential part of business life and business networks seem to be the platforms where lots of business people organize their contacts and try to find new customers, employees or just guys with the same ideas.
being german my first really used network was openbc now named xing and i still have my profile there. however, being in new zealand now i realized that xing is only known by the germans living here which makes the "offline" meetings more a meeting of the german community. nothing bad, just not the thing i expected.
well, some time ago i registered already a profile at linkedin and i get more and more requests to join groups and accept contacts there now so it looks like i have to move my contacts from xing to linkedin. that is a pain and takes time and made me remember the articles that i read over at o'reilly radar about opening the social graph and freeing the data and so on. in fact, i would rather write a "converter" than to do it by hand. probably it is time to write a "meta social network" and that is not the kind of approach that google went with "open social".
a meta network would need to act like a crawler, searching all your networks and contacts and presenting them in a unified way, like google but personalized. you have just one instant messenger for all the im icontacts in all the networks why should you have an hand full of social networks that you check separately? the thing i wonder is the evolution of those networks. will i still update my xing profile in say two years? how many orphaned profiles are in those networks already and how much value are they really?
social networks are great but still a bit beta.

more and more DSL provider in north america decide to break DNS in favour of more revenues from miss typed domain names.

what is that and how does it work?
well, every provider provides his customer with a set of resolving DNS servers. these servers should respond to queries for non existent domain names with a NXDOMAIN that indicates that the domain in question does not exist. instead of that they redirect the customer to a site that has payed links with content "that helps the customer finding what he intended to find".

IP by Verisign or "we had that already"
the intellectual property for that could be in the hands of Verisign who first redirected user to special "help pages" and tried to monetize the miss typed domains. the out cry that days was so loud that it lasted just about 2 days or so. the main difference to the new model is that Verisign did the redirection on the root level for .com and .net. with the DSL providers trying to make money from their customers buy click ads Verisign tries to monetize the root again by selling parts of the root server logfiles. the technique behind their latest product idea in this space is an API where one can proof if a defined not registered domain name got already nameserver traffic. this would change the domain tasting market but as the product will be quite pricey it would be just additional value to some of the top 5 or 10 domainers in the world i guess.
whether Verisign really lounges the product or not and looking at some other developments in how to monetize misspelled domains i have a clear view of infrastructural problems arising out of marketing departments increasing value without knowing the technical drawbacks.

joe armstrong did an awesome job in describing the features of erlang and OTP and since i read the book i am kind of infected. so what happened?
i looked around for jabber servers as i needed one for a corporate setup and found ejabberd again. it is a really stable jabber daemon used in many large scale deployments. the thing that annoyed me every time i used ejabberd was the quite strange config file syntax. instead of being annoyed again i started to dig into it. i did some of the tutorials on the erlang page and tried to figure out what the fuzz about concurrent programming was all about. i have to confess, the language looks strange when you first look at it. the virus, however, infected me and i bought the "programming erlang" book.
after having read this excellent book i started to port my domain related libraries and currently work on a erlang based domain API. it looks promising even though my progress is not that fast as i wished. the language has its advantages but honestly string processing is not one of erlangs strengths. nevertheless i mastered this and have a first rudimentary domain search up and running. nothing fancy but concurrent and on top of OTP.

"Programming Erlang: Software for a Concurrent World" (Joe Armstrong)

a really viral book, i warn every programmer, this could really lead to programming erlang :-)

Technorati Tags: erlang, programming

i am a commited nokia communicator fan and all experiments with other phones ended badly. my first communicator was the almighty 9000i, the oval brick. i used it as my mobile office back in the times in austria. since then i had all the line up till the 9500 (skipped the 9300 of course) but it is aging and the daily use did his part to mae it look a bit rotten over time.
manu has an iPhone now and that made me think. i am always known for beeing a nonconformist and so the iPhone is a bit of a no go thing for me. i looked arround for the current nokias but the reviews of the new models are quite embarrasing. the GPS does not work, the usability is bad, they are slow ... and the models i would like are simply minimum twice the price of an iPhone.
i looked at the openmoko project, looks nice but far from daily use quality. leves you with the HPC phones but they are all windows based with some linux projects on them that all succ more or less. i found a german project for a communicator like device driven by linux, looked nice but has'nt reached release quality even though announced since 2005.

waiting for the google phone. i guess i have to, the SDK for the new google cel phone platform should be available next week and then i hope that there are some early hacks for existing phones soon ... or i have to go with the E90 ... who knows.

this is one of millions of "this is my first post" blog posts.

i finally decidet to write in english after having a german blog that is not much of a vlalue in a english community. for those interested in my german blog, it can be found here.