airbnb.io - Synapse A transparent service discovery framework for connecting an SOA1,830

Airbnb Engineering & Data Science

Airbnb.io Spined HTML

Airbnb Engineering & Data Science Airbnb.io Open SourceEventsBlogGithubCareersAirbnb.io SynapseA transparent service discovery framework for connecting an SOA1,830By Igor SerebryanySynapse is Airbnb's new system for service discovery. Synapse solves the problem of streamlined fail-over in the cloud, where failover via network re-configuration is impossible. The end result is the worthiness to connect internal services together in a scalable, fault-tolerant way. Motivation Synapse emerged from the need to maintain high-availability applications in the cloud. Traditional high-availability techniques, which involve using a CRM like pacemaker, do not work in environments where the end-user has no tenancy over the networking. In an environment like Amazon's EC2, all of the misogynist workarounds are suboptimal: Round-robin DNS: Slow to converge, and doesn't work when applications enshroud DNS lookups (which is frequent) Elastic IPs: slow to converge, limited in number, public-facing-only, which makes them less useful for internal services ELB: Again, public-facing only, and only useful for HTTP One solution to this problem is a discovery service, like Apache Zookeeper. However, Zookeeper and similar services have their own problems: Service discovery is embedded in all of your apps; often, integration is not simple The discovery layer itself is subject to failure Requires spare servers/instances Synapse solves these difficulties in a simple and fault-tolerant way. How Synapse Works Synapse runs on your using servers; here at Airbnb, we just run it on every box we deploy. The heart of synapse is unquestionably HAProxy, a stable and proven routing component. For every external service that your using talks to, we assign a synapse local port on localhost. Synapse creates a proxy from the local port to the service, and you reconfigure your using to talk to the proxy. Synapse comes with a number of watchers, which are responsible for service discovery. The synapse watchers take superintendency of re-configuring the proxy so that it unchangingly points at misogynist servers. We've included a number of default watchers, including ones that query zookeeper and ones using the AWS API. It is easy to write your own watchers for your use case, and we encourage submitting them when to the project. Example Migration Let's suppose your rails using depends on a Postgres database instance. The database.yaml file has the DB host and port hardcoded: production: database: mydb host: mydb.example.com port: 5432 You would like to be worldly-wise to goof over to a variegated database in specimen the original dies. Let's suppose your instance is running in AWS and you're using the tag 'proddb' set to 'true' to indicate the prod DB. You set up synapse to proxy the DB connection on localhost:3219 in the synapse.conf.yaml file. Add a hash under services that looks like this: --- services: proddb: default_servers: - name: "default-db" host: "mydb.example.com" port: 5432 discovery: method: "awstag" tag_name: "proddb" tag_value: "true" haproxy: port: 3219 server_options: "check inter 2000 rise 3 fall 2" frontend: mode tcp backend: mode tcp And then transpiration your database.yaml file to squint like this: production: database: mydb host: localhost port: 3219 Start up synapse. It will configure HAProxy with a proxy from localhost:3219 to your DB. It will struggle to find the DB using the AWS API; if that does not work, it will default to the DB given in default_servers. In the worst case, if AWS API is lanugo and you need to transpiration which DB your using talks to, simply edit the synapse.conf.json file, update the default_servers and restart synapse. HAProxy will be transparently reloaded, and your using will alimony running without a hiccup.LinksGithub© Airbnb, Inc.© Airbnb, Inc.CareersDesignCareersDesignCareersDesign