Amazon DynamoDB is a fully managed NoSQL database service that provides fast performance at any scale. Today’s web-based applications often encounter database scaling challenges when faced with growth in users, traffic, and data. With Amazon DynamoDB, developers scaling cloud-based applications can start small with just the capacity they need and then increase the request capacity of a given table as their app grows in popularity. Their tables can also grow without limits as their users store increasing amounts of data. Behind the scenes, Amazon DynamoDB automatically spreads the data and traffic for a table over a sufficient number of servers to meet the request capacity specified by the customer. Amazon DynamoDB offers low, predictable latencies at any scale. Customers can typically achieve average service-side in the single-digit milliseconds. Amazon DynamoDB stores data on Solid State Drives (SSDs) and replicates it synchronously across multiple AWS Availability Zones in an AWS Region to provide built-in high availability and data durability.

History of NoSQL at Amazon – Dynamo

The Amazon.com ecommerce platform consists of hundreds of decoupled services developed and managed in a decentralized fashion. Each service encapsulates its own data and presents a hardened API for others to use. Most importantly, direct database access to the data from outside its respective service is not allowed. This architectural pattern was a response to the scaling challenges that had challenged Amazon.com through its first 5 years, when direct database access was one of the major bottlenecks in scaling and operating the business. While a service-oriented architecture addressed the problems of a centralized database architecture, each service was still using traditional data management systems. The growth of Amazon’s business meant that many of these services needed more scalable database solutions.

In response, we began to develop a collection of storage and database technologies to address the demanding scalability and reliability requirements of the Amazon.com ecommerce platform. We had been pushing the scalability of commercially available technologies to their limits and finally reached a point where these third party technologies could no longer be used without significant risk. This was not our technology vendors’ fault; Amazon’s scaling needs were beyond the specs for their technologies and we were using them in ways that most of their customers were not. A number of outages at the height of the 2004 holiday shopping season can be traced back to scaling commercial technologies beyond their boundaries.

Dynamo was born out of our need for a highly reliable, ultra-scalable key/value database. This non-relational, or NoSQL, database was targeted at use cases that were core to the Amazon ecommerce operation, such as the shopping cart and session service. Any downtime or performance degradation in these services has an immediate financial impact and their fault-tolerance and performance requirements for their data systems are very strict. These services also require the ability to scale infrastructure incrementally to accommodate growth in request rates or dataset sizes. Another important requirement for Dynamo was predictability. This is not just predictability of median performance and latency, but also at the end of the distribution (the 99.9th percentile), so we could provide acceptable performance for virtually every customer.

To achieve all of these goals, we needed to do groundbreaking work. After the successful launch of the first Dynamo system, we documented our experiences in a paper so others could benefit from them. Since then, several Dynamo clones have been built and the Dynamo paper has been the basis for several other types of distributed databases. This demonstrates that Amazon is not the only company than needs better tools to meet their database needs.

Lessons learned from Amazon’s Dynamo

Dynamo has been in use by a number of core services in the ecommerce platform, and their engineers have been very satisfied by its performance and incremental scalability. However, we never saw much adoption beyond these core services. This was remarkable because although Dynamo was originally built to serve the needs of the shopping cart, its design and implementation were much broader and based on input from many other service architects. As we spoke to many senior engineers and service owners, we saw a clear pattern start to emerge in their explanations of why they didn’t adopt Dynamo more broadly: while Dynamo gave them a system that met their reliability, performance, and scalability needs, it did nothing to reduce the operational complexity of running large database systems. Since they were responsible for running their own Dynamo installations, they had to become experts on the various components running in multiple data centers. Also, they needed to make complex tradeoff decisions between consistency, performance, and reliability. This operational complexity was a barrier that kept them from adopting Dynamo.

During this period, several other systems appeared in the Amazon ecosystem that did meet their requirements for simplified operational complexity, notably Amazon S3 and Amazon SimpleDB. These were built as managed web services that eliminated the operational complexity of managing systems while still providing extremely high durability. Amazon engineers preferred to use these services instead of managing their own databases like Dynamo, even though Dynamo’s functionality was better aligned with their applications’ needs.

With Dynamo we had taken great care to build a system that met the requirements of our engineers. After evaluations, it was often obvious that Dynamo was ideal for many database use cases. But … we learned that engineers found the prospect of running a large software system daunting and instead looked for less ideal design alternatives that freed them from the burden of managing databases and allowed them to focus on their applications.

It became obvious that developers strongly preferred simplicity to fine-grained control as they voted “with their feet” and adopted cloud-based AWS solutions, like Amazon S3 and Amazon SimpleDB, over Dynamo. Dynamo might have been the best technology in the world at the time but it was still software you had to run yourself. And nobody wanted to learn how to do that if they didn’t have to. Ultimately, developers wanted a service.

History of NoSQL at Amazon – SimpleDB

One of the cloud services Amazon developers preferred for their database needs was Amazon SimpleDB. In the 5 years that SimpleDB has been operational, we have learned a lot from its customers.

First and foremost, we have learned that a database service that takes away the operational headache of managing distributed systems is extremely powerful. Customers like SimpleDB’s table interface and its flexible data model. Not having to update their schemas when their systems evolve makes life much easier. However, they most appreciate the fact that SimpleDB just works. It provides multi-data center replication, high availability, and offers rock-solid durability. And yet customers never need to worry about setting up, configuring, or patching their database.

Second, most database workloads do not require the complex query and transaction capabilities of a full-blown relational database. A database service that only presents a table interface with a restricted query set is a very important building block for many developers.

While SimpleDB has been successful and powers the applications of many customers, it has some limitations that customers have consistently asked us to address.

Domain scaling limitations. SimpleDB requires customers to manage their datasets in containers called Domains, which have a finite capacity in terms of storage (10 GB) and request throughput. Although many customers worked around SimpleDB’s scaling limitations by partitioning their workloads over many Domains, this side of SimpleDB is certainly not simple. It also fails to meet the requirement of incremental scalability, something that is critical to many customers looking to adopt a NoSQL solution.

Predictability of Performance. SimpleDB, in keeping with its goal to be simple, indexes all attributes for each item stored in a domain. While this simplifies the customer experience on schema design and provides query flexibility, it has a negative impact on the predictability of performance. For example, every database write needs to update not just the basic record, but also all attribute indices (regardless of whether the customer is using all the indices for querying). Similarly, since the Domain maintains a large number of indices, its working set does not always fit in memory. This impacts the predictability of a Domain’s read latency, particularly as dataset sizes grow.

Consistency. SimpleDB’s original implementation had taken the “eventually consistent” approach to the extreme and presented customers with consistency windows that were up to a second in duration. This meant the system was not intuitive to use and developers used to a more traditional database solution had trouble adapting to it. The SimpleDB team eventually addressed this issue by enabling customers to specify whether a given read operation should be strongly or eventually consistent.

Pricing complexity. SimpleDB introduced a very fine-grained pricing dimension called “Machine Hours.” Although most customers have eventually learned how to predict their costs, it was not really transparent or simple.

Introducing DynamoDB

As we thought about how to address the limitations of SimpleDB and provide 1) the most scalable NoSQL solution available and 2) predictable high performance, we realized our goals could not be met with the SimpleDB APIs. Some SimpleDB operations require that all data for a Domain is on a single server, which prevents us from providing the seamless scalability our customers are demanding. In addition, SimpleDB APIs assume all item attributes are automatically indexed, which limits performance.

We concluded that an ideal solution would combine the best parts of the original Dynamo design (incremental scalability, predictable high performance) with the best parts of SimpleDB (ease of administration of a cloud service, consistency, and a table-based data model that is richer than a pure key-value store). These architectural discussions culminated in Amazon DynamoDB, a new NoSQL service that we are excited to release today.

Amazon DynamoDB is based on the principles of Dynamo, a progenitor of NoSQL, and brings the power of the cloud to the NoSQL database world. It offers customers high-availability, reliability, and incremental scalability, with no limits on dataset size or request throughput for a given table. And it is fast – it runs on the latest in solid-state drive (SSD) technology and incorporates numerous other optimizations to deliver low latency at any scale.

Amazon DynamoDB is the result of everything we’ve learned from building large-scale, non-relational databases for Amazon.com and building highly scalable and reliable cloud computing services at AWS. Amazon DynamoDB is a NoSQL database service that offers the following benefits:

• Managed. DynamoDB frees developers from the headaches of provisioning hardware and software, setting up and configuring a distributed database cluster, and managing ongoing cluster operations. It handles all the complexities of scaling and partitions and re-partitions your data over more machine resources to meet your I/O performance requirements. It also automatically replicates your data across multiple Availability Zones (and automatically re-replicates in the case of disk or node failures) to meet stringent availability and durability requirements. From our experience of running Amazon.com, we know that manageability is a critical requirement. We have seen many job postings from companies using NoSQL products that are looking for NoSQL database engineers to help scale their installations. We know from our Amazon experiences that once these clusters start growing, managing them becomes the same nightmare that running large RDBMS installations was. Because Amazon DynamoDB is a managed service, you won’t need to hire experts to manage your NoSQL installation—your developers can do it themselves.

• Scalable. Amazon DynamoDB is designed to scale the resources dedicated to a table to hundreds or even thousands of servers spread over multiple Availability Zones to meet your storage and throughput requirements. There are no pre-defined limits to the amount of data each table can store. Developers can store and retrieve any amount of data and DynamoDB will spread the data across more servers as the amount of data stored in your table grows.

• Fast. Amazon DynamoDB provides high throughput at very low latency. It is also built on Solid State Drives to help optimize for high performance even at high scale. Moreover, by not indexing all attributes, the cost of read and write operations is low as write operations involve updating only the primary key index thereby reducing the latency of both read and write operations. An application running in EC2 will typically see average service-side latencies in the single-digit millisecond range for a 1KB object. Most importantly, DynamoDB latencies are predictable. Even as datasets grow, latencies remain stable due to the distributed nature of DynamoDB’s data placement and request routing algorithms.

• Durable and Highly Available. Amazon DynamoDB replicates its data over at least 3 different data centers so that the system can continue to operate and serve data even under complex failure scenarios.

• Flexible. Amazon DynamoDB is an extremely flexible system that does not force its users into a particular data model or a particular consistency model. DynamoDB tables do not have a fixed schema but instead allow each data item to have any number of attributes, including multi-valued attributes. Developers can optionally use stronger consistency models when accessing the database, trading off some performance and availability for a simpler model. They can also take advantage of the atomic increment/decrement functionality of DynamoDB for counters.

• Low cost. Amazon DynamoDB’s pricing is simple and predictable: Storage is $1 per GB per month. Requests are priced based on how much capacity is reserved: $0.01 per hour for every 10 units of Write Capacity and $0.01 per hour for every 50 units of Read Capacity. A unit of Read (or Write) Capacity equals one read (or write) per second of capacity for items up to 1KB in size. If you use eventually consistent reads, you can achieve twice as many reads per second for a given amount of Read Capacity. Larger items will require additional throughput capacity.

In the current release, customers will have the choice of using two types of keys for primary index querying: Simple Hash Keys and Composite Hash Key / Range Keys:

Simple Hash Key gives DynamoDB the Distributed Hash Table abstraction. The key is hashed over the different partitions to optimize workload distribution. For more background on this please read the original Dynamo paper.

Composite Hash Key with Range Key allows the developer to create a primary key that is the composite of two attributes, a “hash attribute” and a “range attribute.” When querying against a composite key, the hash attribute needs to be uniquely matched but a range operation can be specified for the range attribute: e.g. all orders from Werner in the past 24 hours, all log entries from server 16 with clients IP addresses on subnet 192.168.1.0

Performance Predictability in DynamoDB

In addition to taking the best ideas of Dynamo and SimpleDB, we have added new functionality to provide even greater performance predictability.

Cloud-based systems have invented solutions to ensure fairness and present their customers with uniform performance, so that no burst load from any customer should adversely impact others. This is a great approach and makes for many happy customers, but often does not give a single customer the ability to ask for higher throughput if they need it.

As satisfied as engineers can be with the simplicity of cloud-based solutions, they would love to specify the request throughput they need and let the system reconfigure itself to meet their requirements. Without this ability, engineers often have to carefully manage caching systems to ensure they can achieve low-latency and predictable performance as their workloads scale. This introduces complexity that takes away some of the simplicity of using cloud-based solutions.

The number of applications that need this type of performance predictability is increasing: online gaming, social graphs applications, online advertising, and real-time analytics to name a few. AWS customers are building increasingly sophisticated applications that could benefit from a database that can give them fast, predictable performance that exactly matches their needs.

Amazon DynamoDB’s answer to this problem is “Provisioned Throughput.” Customers can now specify the request throughput capacity they require for a given table. Behind the scenes, DynamoDB will allocate sufficient resources to the table to predictably achieve this throughput with low-latency performance. Throughput reservations are elastic, so customers can increase or decrease the throughput capacity of a table on-demand using the AWS Management Console or the DynamoDB APIs. CloudWatch metrics enable customers to make informed decisions about the right amount of throughput to dedicate to a particular table. Customers using the service tell us that it enables them to achieve the appropriate amount of control over scaling and performance while maintaining simplicity. Rather than adding server infrastructure and re-partitioning their data, they simply change a value in the management console and DynamoDB takes care of the rest.

Summary

Amazon DynamoDB is designed to maintain predictably high performance and to be highly cost efficient for workloads of any scale, from the smallest to the largest internet-scale applications. You can get started with Amazon DynamoDB using a free tier that enables 40 million of requests per month free of charge. Additional request capacity is priced at cost-efficiently hourly rates as low as $.01 per hour for 10 units of Write Capacity or 50 strongly consistent units of Read Capacity (if you use eventually consistent reads you can get twice the throughput at the same cost, or the same read throughput at half the cost) Also, replicated solid state disk (SSD) storage is $1 per GB per month. Our low request pricing is designed to meet the needs of typical database workloads that perform large numbers of reads and writes against every GB of data stored.

To learn more about Amazon DynamoDB its functionality, APIs, use cases, and service pricing, please visit the detail page at aws.amazon.com/DynamoDB and also the Developer Guide. I am excited to see the years of experience with systems such as Amazon Dynamo result in an innovative database service that can be broadly used by all our customers.

View full post on All Things Distributed

Percona Server with XtraDB for Software-as-a-Service Application Databases is posted on the white-paper section of our website. No registration is required; it is just a PDF, freely downloadable with no marketing follow-up. Many thanks to Mark Callaghan and Dimitri Kravtchuk, who reviewed and gave valuable suggestions that made the white paper better.

Entry posted by Baron Schwartz |
No comment

Add to: | | | |

View full post on MySQL Performance Blog

Read more on Business Wire via Yahoo! Finance

Here at LinkedIn we need to build a variety of services that can horizontally scale.  Two of the common reasons we need to horizontally scale is because we need either more memory or more CPUs (and sometimes both) than we can fit into a single machine.  The solution to this problem is often simple, one cluster of computers all running the same service to do the processing and another cluster of memcached instances to store the data.  Or, maybe instead of memcached, you shard your data across multiple database servers.  Add a basic round robin load balancer and you’re done!

The problem with this approach is now your data and the processing occur in two different systems.  That means every incoming request needs to go to memcached or the database, retrieve the data and then perform the processing.  When retrieving the data takes too long, or when you need to perform some complex transformation of that data before you can start processing, you have a problem.  To solve this problem you decide to cache the data in the same machine processing that data.  You grab something like ehcache or maybe build your own solution and have your in process cache.

But now your round robin load balancer doesn’t work.  If you just send requests randomly to different machines your local cache won’t be used.  Or, even worse, you’ll end up trying to cache all the data on every instance.  What you really want is a load balancer that knows what instances of the service are available to server traffic and what data they have locally available.

This is the exact situation we ended up with here in our search engine.  Our search index is simply too large for a single machine to execute a query fast enough.  So you split the index into multiple smaller indexes and have multiple searchers to execute queries against those sub-indexes.  To satisfy a request a broker, in parallel, has each searcher execute the query and then combines the sub-result sets into the final results.  In order to do it’s job the broker needs to know all of the searchers and which sub-index they have on them.  Keeping track of this information is commonly known as Group Management and we wrote a generic library call Norbert which provides the functionality.

Group Management with Norbert

Group management can mean different things to different people so for our purposes we’ll say that such a system at least:

1. Maintains a record of all the servers in the group
2. Allows servers to come and go dynamically
3. Notifies clients and other group members when a server comes or goes

Several applications in the SNA team here at LinkedIn needed group management so we extracted that functionality from our horizontally distributed social graph engine, generalized it and packaged it up as our open source Norbert project.  In the rest of this article I’m going to describe how Norbert defines a group (or cluster) and how to use the ClusterClient interface to add group management to your own application.

Centrally Managed Cluster Metadata

First you can (optionally) specify partition ids.  Partition ids can be used to create “virtual” partitions, partitions that don’t exist by themselves but are co-located with other partitions.  Some consistent hash strategies use virtual partitions to more evenly balance data or load in a system, however you do not need to define partition ids if they are not useful for your application.  Whether you use partitions ids or not the next higher entity in a Norbert cluster is a Node.  A Node is defined by three pieces of data:

1. An arbitrary integer id.  Norbert doesn’t place any specific meaning on this id so you can allocate them to Nodes in whatever way makes the most sense for your application or deployment strategy.
2. A string, which is called “url.”  Norbert’s group management features don’t place any specific meaning on this string but, as it’s name suggests, it’s meant to be used to locate and communicate with Nodes over a network.  It’s format will likely be dictated by the technology you use for client/server communication.
3. The partition ids assigned to the Node, if you are using partition ids.

Finally you specify the name for the set of Nodes that make up your cluster.  Norbert persistently stores all of the metadata for the Nodes within a given cluster centrally in ZooKeeper.

Node Availability

If the metadata about the Nodes is persistently stored in ZooKeeper how do you know what subset of the Nodes is currently available to service requests?  A second tree of ephemeral znodes is kept in ZooKeeper which tracks the availability of each individual Node in the cluster.  The availability data, rather than being centrally managed by a system administrator, is handled directly by your software. Once your application has finished starting up you tell Norbert that you’re ready to handle requests.  So why not just have the application specify both the Node metadata and the Node availability in a single ephemeral znode in ZooKeeper?  The reason is to avoid a rogue service on your network from causing problems.  Perhaps you are in the middle of setting up a new system and accidentally start your application.  You likely don’t want it to start receiving traffic yet, and this mechanism prevents that from happening.  A Node isn’t considered online and ready to handle traffic until both it’s metadata is defined and it’s available znode created.

The ClusterClient API

To add group management to your application using Norbert requires interacting with a single interface called ClusterClient.  There are currently two implementations of ClusterClient, an in memory only client for testing purposes and a client which uses ZooKeeper.  Once you’ve instantiated your ClusterClient you will find APIs to create nodes, delete nodes, mark a particular node available or unavailable, etc.  But there is one specific API that I’d like to focus on:

ClusterListenerKey addListener(ClusterListener listener);

The ClusterListener interface is defined as:

public interface ClusterListener {
/**
* Handle the case that you are now connected to the cluster.
*
* @param nodes the current list of available Nodes stored in the cluster metadata
*/
void handleClusterConnected(Set nodes);

/**
* Handle the case that the cluster topology has changed.
*
* @param nodes the current list of availableNodes stored in the cluster metadata
*/
void handleClusterNodesChanged(Set nodes);

/**
* Handle the case that the cluster is now disconnected.
*/
void handleClusterDisconnected();

/**
* Handle the case that the cluster is now shutdown.
*/
void handleClusterShutdown();
}


Every registered ClusterListener is notified by Norbert whenever the group membership changes, you are connected or disconnected from ZooKeeper and when the ClusterClient is shut down.  And that is all you really need to do to add group management capabilities to your application using Norbert, implement a ClusterListener and have it update your application’s routing tables whenever there are membership changes.

View full post on SNA Projects Blog

Read more on Marketwire via Yahoo! Finance

Read more on Marketwire via Yahoo! Finance

Equipping an office with the standard Windows applications is an expensive activity. Some Microsoft applications make you purchase the software itself and then pay additional licensing fees when you connect your Microsoft client computer to a Microsoft server computer.

Damn Small Linux applications are free. Some of them are included with the initial download while others have to be downloaded and installed subsequently. There are several reasons for this seemingly intricate procedure. Not everybody wants to use the same applications. Furthermore, the very strict selection of the applications in its base package helps the Damn Small Linux distribution to respect the hard to believe 50 Megabyte limit. Once you have installed Damn Small Linux you can download Open Source (free) versions of Office and run them under Linux. Please note that you can download and run these programs under Microsoft Windows as well.

All work and no play makes Jack a dull boy. Well Jack doesn’t have to worry if he’s running Damn Small Linux. It comes with lots of free games. Somehow we have the impresssion that you will not need any tutorials to get the games running.

Some Linux applications are not only free but are actually superior when compared to the corresponding Windows applications. For example, have you ever heard of an e-mail client called Sylpheed? This application is part of the basic Damn Small Linux distribution. It has several advantages over those extremely popular (perhaps we should say unpopular) Microsoft e-mail offerings, Outlook and Outlook Express. Sylpheed handles spam much better than do its Windows counterparts. And Sypheed allows you to process threads of e-mail messages. For example, you correspond with several people about wholesale wine purchases. Sypheed makes it easy to process those messages while ignoring unrelated ones.

Since we’re talking about email, what about it’s cousin the Internet? That will be the subject of our next article in this series.

Enterprise 2.0 is defined as: “a system of web-based technologies that provide rapid and agile collaboration, information sharing, emergence and integration capabilities in the extended enterprise” . Basically, it’s about user-friendly web applications that let employees and partners of the company collaborate and generate ideas quickly. The proposed benefits are that this will result in open innovation and improve the company’s processes and products.

In modern enterprises employees are generating lots of content such as blog posts, articles, article comments, wiki content, newsletter broadcasts and discussion board replies which contain valuable knowledge. Enterprise 2.0 is about linking all this content together and making it easily available to employees. In enterprise 2.0 information will be finding the user instead of the other way around.

Employees should be able to have discussions about the content online, generating ideas which in turn lead to innovation of improved processes, as well as new products and services. Letting employees directly communicate and share ideas outside conventional meetings will result in much more ideas being generated.

Tagging is a buzzword today and an essential feature in both social networks and enterprise 2.0. Tagging allows for grouping users together with other users and content through automatic recommendations and suggestions to the end user.Closely related to tagging of data is enterprise bookmarking which commonly is part of enterprise 2.0. Using enterprise bookmarking employees can tag, store and organize their bookmarked external as well as internal sources of information and also share it with their co-workers.

Enterprise 2.0 presents companies with a wide range of benefits including tracking ideas, attitudes and opinions; innovation; knowledge sharing; and peer-based project planning.
With APIs being provided by an increasing number of web applications, it is possible to linking your enterprise 2.0 system together with other third party web services, creating a mash-up which combines data from multiple sources. That is something that will make the enterprise 2.0 system even more useful and valuable to its end-users. It will allow you to bring in third party news, content, maps and more seamlessly without having your end-users leave your company’s enterprise 2.0 application.

Using PHP and MySQL as development platform

Enterprise 2.0 web applications are very similar to social networks in terms of functionality. When it comes to social networks PHP and MySQL is a frequently used combination of programming language and database. Lots of most popular custom business softwares and web applications already built by this technologies. Facebook for example, is developed on this platform.

Introduction

You’ve just started a new job, you need to hit the ground running and quickly, but the rest of the team are “too busy” with their work and you don’t want to keep pestering your new colleagues or boss with questions. Enter the MySQL Binary Log.

If you have never heard of the Binary Log I suggest at least finding out what it is and what it is usually used for before you continue with the tutorial.

Tools of the trade

O.K. Let’s start with the tools required for the job. You will need to download a neat little toolkit called unxutils from Source Forge and make sure it’s install path is referenced in your Path. Unxutils is a handy tool which brings many Unix commands to the windows desktop. It especially useful if you commonly switch between Windows and Linux platforms. Once you have done this you need to open a Command Prompt window and test to make sure you can access the commands. Type the following command and hit enter.

ls -l

If you get a directory listing you can happily move on to the next step. Enabling the Binary Log on Windows.  The above command was simply to test if unxutils was installed correctly.

The Service

If you search Google for “Enabling MySQL Binary Log on Windows” you will get a stack of results returned some of which will lead you to MySQL bug status results. The reason for this is that since most Windows users will have MySQL installed, using a Windows installer, on their system as a Service and by default Binary Logging is not enabled. If you follow the below instructions to the letter you will be one of few who has Binary Logging enabled on a Windows box.

For this example I am going to install another MySQL Service along side my original one as I do not wish to use Binary Logging all the time. If you already have a MySQL Service running make sure you stop it at this point. If you don’t already have the command line open from the first task then fire it up and type the following command.

sc create MySQLBinLogging binPath= \”\”C:\\Path\\To\\MySQL Server\\bin\\mysqld\” –log-bin=yourmachinename MySQL\”

After you have run this command you should see a success message at which point you can open the services manager and start the service. On a typical MySQL installation the binary log file will be located in the MySQL data directory on Vista this is typically in \“C:\\Program\\Data\\MySQL Server\\data\” and will be named as you specified with the –log-bin directive when installing the service. The log file is rotated on a size basis so the first log file should have a suffix of 000001 and so on. At this point we are ready to extract the juicy data we have been waiting to get at.

The Data

Again we will be operating from the command line for now so assuming you still have it open change your path to the data directory where the log file is type the following command and hit enter.

tail -f mylogfile

And that is it. If you want to output the data into a text file for further analysis just type the following.

tail -f mylogfile > myoutputfile.txt

NOTE: The binary log only stored update data so you won’t get your select data here.  You will need to work with the General Query Log for selects.