Read more on PR Newswire via Yahoo! Finance

-Lee

View full post on DrizzleBlog

Please make sure and sign up at http://drizzle.org/wiki/Drizzle_Developer_Day_2010_signup. Space will be limited and it is filling up fast.

Also, to make this day a success, please provide your input on discussion topics at http://drizzle.org/wiki/Drizzle_Developer_Day_2010

Hope to see you there!

View full post on DrizzleBlog

First, how did it come about that I started working on the storage engine API?

From Commands to Transactions

Well, it really goes back to my work on Drizzle’s replication system.  I had implemented a simple, fast, and extensible log which stored records of the data changes made to a server.  Originally, the log was called the Command Log, because the Google Protobuffer messages it contained were called message::Commands.  The API  for implementing replication plugins was very simple and within a month or so of debuting the API, quite a few replication plugins had been built, including one replicating to Memcached, a prototype one replicating to Gearman, and a filtering replicator plugin.

In addition, Marcus Eriksson had created the RabbitReplication project which could replicate from Drizzle to other data stores, including Cassandra and Project Voldemort.  However, Marcus did not actually implement any C/C++ plugins using the Drizzle replication API.  Instead, RabbitReplication simply read the new Command Log, which due to it simply being a file full of Google Protobuffer messages, was quick and easy to read into memory using a variety of different programming languages.  RabbitReplication is written in Java, and it was great to see other programming languages be able to read Drizzle’s replication log so easily.  Marcus later coded up a C++ TransactionApplier plugin which replaces the Drizzle replication log and instead replicates the GPB messages directly to RabbitMQ.

And there, you’ll note that one of the plugins involved in Drizzle’s replication system is called TransactionApplier.  It used to be called CommandApplier. That was because the GPB Command messages were individual row change events for the most part.  However, I made a series of changes to the replication API and now the GPB messages sent through the APIs are of class message::Transaction.  message::Transaction objects contain a transaction context, with information about the transaction’s start and end time, it’s transaction identifer, along with a series of message::Statement objects, each of which representing a part of the data changes that the SQL transaction made.

Thus, the Command Log now turned into the Transaction Log, and everywhere the term Command was used now was replaced with the terms Transaction and Statement (depending on whether you were talking about the entire Transaction or a piece of it). Log entries were now written at COMMIT to the Transaction Log and were not written if no COMMIT occurred¹.

After finishing this work to make the transaction log write Transaction messages at commit time, I was keen to begin coding up the publisher and subscriber plugins which represent a node in the replication environment. However, Brian had asked me to delay working on other replication features and ensure that the replication API could support fully distributed transactions via the X/Open XA distributed transaction protocol. XA support had been removed from Drizzle when the MySQL binlog and original replication system was ripped out and needed some TLC. Fair enough, I said. So, off I went to work on XA.

If Only It Were Simple…

As anyone who has worked on the MySQL source code or developed storage engines for MySQL knows, working with the MySQL pluggable storage engine API is sometimes not the easiest or most straightforward thing. I think the biggest problem with the MySQL storage engine API is that, due to understandable historical reasons, it’s an API that was designed with the MyISAM and HEAP storage engines in mind. Much of the transactional pieces of the API seem to be a bolted-on afterthought and can be very confusing to work with.

As an example, Paul McCullagh, developer of the transactional storage engine PBXT, recently emailed the mysql internals mailing list asking how the storage engine could tell when a SQL statement started and ended. You would think that such a seemingly basic functionality would have a simple answer. You’d be wrong. Monty Widenius answered like this:

Why not simply have a counter in your transaction object for how start_stmt – reset(); When this is 0 then you know stmnt ended.

In Maria we count number of calls to external_lock() and when the sum goes to 0 we know the transaction has ended.

To this, Mark Callaghan responded:

Why does the solution need to be so obscure?

Monty answered (emphasis mine):

Historic reasons.

MySQL never kept a count of which handlers are used by a transaction, only which tables.

So the original logic was that external_lock(lock/unlock) is called for each usage of the table, which is normally more than enough information for a handler to know when a statement starts/ends.

The one case this didn’t work was in the case someone does lock tables as then external_lock is not called per statement. It was to satisfy this case that we added a call to start_stmt() for each table.

It’s of course possible to change things so that start_stmt() / end_stmt() would be called once per used handler, but this would be yet another overhead for the upper level to do which the current handlers that tracks call to external_lock() doesn’t need.

Well, in Drizzle-land, we aren’t beholden to “historic reasons” So, after looking through the in-need-of-attention transaction processing code in the kernel, I decided that I would clean up the API so that storage engines did not have to jump through hoops to notify the kernel they participate in a transaction or just to figure out when a statement and a transaction started and ended.

The resulting changes to the API are quite dramatic I think, but I’ll leave it to the storage engine developers to tell me if the changes are good or not. The following is a summary of the changes to the storage engine API that I committed in the last few weeks.

plugin::StorageEngine Split Into Subclasses

The very first thing I did was to split the enormous base plugin class for a storage engine, plugin::StorageEngine, into two other subclasses containing transactional elements. plugin::TransactionalStorageEngine is now the base class for all storage engines which implement SQL transactions:

/**
 * A type of storage engine which supports SQL transactions.
 *
 * This class adds the SQL transactional API to the regular
 * storage engine.  In other words, it adds support for the
 * following SQL statements:
 *
 * START TRANSACTION;
 * COMMIT;
 * ROLLBACK;
 * ROLLBACK TO SAVEPOINT;
 * SET SAVEPOINT;
 * RELEASE SAVEPOINT;
 */
class TransactionalStorageEngine :public StorageEngine
{
public:
  TransactionalStorageEngine(const std::string name_arg,
                             const std::bitset<HTON_BIT_SIZE> &flags_arg= HTON_NO_FLAGS);
 
  virtual ~TransactionalStorageEngine();
...
private:
  void setTransactionReadWrite(Session& session);
 
  /*
   * Indicates to a storage engine the start of a
   * new SQL transaction.  This is called ONLY in the following
   * scenarios:
   *
   * 1) An explicit BEGIN WORK/START TRANSACTION is called
   * 2) After an explicit COMMIT AND CHAIN is called
   * 3) After an explicit ROLLBACK AND RELEASE is called
   * 4) When in AUTOCOMMIT mode and directly before a new
   *    SQL statement is started.
   */
  virtual int doStartTransaction(Session *session, start_transaction_option_t options)
  {
    (void) session;
    (void) options;
    return 0;
  }
 
  /**
   * Implementing classes should override these to provide savepoint
   * functionality.
   */
  virtual int doSetSavepoint(Session *session, NamedSavepoint &savepoint)= 0;
  virtual int doRollbackToSavepoint(Session *session, NamedSavepoint &savepoint)= 0;
  virtual int doReleaseSavepoint(Session *session, NamedSavepoint &savepoint)= 0;
 
  /**
   * Commits either the "statement transaction" or the "normal transaction".
   *
   * @param[in] The Session
   * @param[in] true if it's a real commit, that makes persistent changes
   *            false if it's not in fact a commit but an end of the
   *            statement that is part of the transaction.
   * @note
   *
   * 'normal_transaction' is also false in auto-commit mode where 'end of statement'
   * and 'real commit' mean the same event.
   */
  virtual int doCommit(Session *session, bool normal_transaction)= 0;
 
  /**
   * Rolls back either the "statement transaction" or the "normal transaction".
   *
   * @param[in] The Session
   * @param[in] true if it's a real commit, that makes persistent changes
   *            false if it's not in fact a commit but an end of the
   *            statement that is part of the transaction.
   * @note
   *
   * 'normal_transaction' is also false in auto-commit mode where 'end of statement'
   * and 'real commit' mean the same event.
   */
  virtual int doRollback(Session *session, bool normal_transaction)= 0;
  virtual int doReleaseTemporaryLatches(Session *session)
  {
    (void) session;
    return 0;
  }
  virtual int doStartConsistentSnapshot(Session *session)
  {
    (void) session;
    return 0;
  }
};

As you can see, plugin::TransactionalStorageEngine inherits from plugin::StorageEngine and extends it with a series of private pure virtual methods that implement the SQL transaction parts of a query — doCommit(), doRollback(), etc. Implementing classes simply inherit from plugin::TransactionalStorageEngine and implement their internal transaction processing in these private methods.

In addition to the SQL transaction, however, is the concept of an XA transaction, which is for distributed transaction coordination. The XA protocol is a two-phase commit protocol because it implements a PREPARE step before a COMMIT occurs. This XA API is exposed via two other classes, plugin::XaResourceManager and plugin::XaStorageEngine. plugin::XaResourceManager derived classes implement the resource manager API of the XA protocol. plugin::XaStorageEngine is a storage engine subclass which, while also implementing SQL transactions, also implements XA transactions.

Here is the plugin::XaResourceManager class:

/**
 * An abstract interface class which exposes the participation
 * of implementing classes in distributed transactions in the XA protocol.
 */
class XaResourceManager
{
public:
  XaResourceManager() {}
  virtual ~XaResourceManager() {}
...
private:
  /**
   * Does the COMMIT stage of the two-phase commit.
   */
  virtual int doXaCommit(Session *session, bool normal_transaction)= 0;
  /**
   * Does the ROLLBACK stage of the two-phase commit.
   */
  virtual int doXaRollback(Session *session, bool normal_transaction)= 0;
  /**
   * Does the PREPARE stage of the two-phase commit.
   */
  virtual int doXaPrepare(Session *session, bool normal_transaction)= 0;
  /**
   * Rolls back a transaction identified by a XID.
   */
  virtual int doXaRollbackXid(XID *xid)= 0;
  /**
   * Commits a transaction identified by a XID.
   */
  virtual int doXaCommitXid(XID *xid)= 0;
  /**
   * Notifies the transaction manager of any transactions
   * which had been marked prepared but not committed at
   * crash time or that have been heurtistically completed
   * by the storage engine.
   *
   * @param[out] Reference to a vector of XIDs to add to
   *
   * @retval
   *  Returns the number of transactions left to recover
   *  for this engine.
   */
  virtual int doXaRecover(XID * append_to, size_t len)= 0;
};

and here is the plugin::XaStorageEngine class:

/**
 * A type of storage engine which supports distributed
 * transactions in the XA protocol.
 */
class XaStorageEngine :public TransactionalStorageEngine,
                       public XaResourceManager
{
public:
  XaStorageEngine(const std::string name_arg,
                  const std::bitset<HTON_BIT_SIZE> &flags_arg= HTON_NO_FLAGS);
 
  virtual ~XaStorageEngine();
  ...
};

Pretty clear. A plugin::XaStorageEngine inherits from both plugin::TransactionStorageEngine and plugin::XaResourceManager because it implements both SQL transactions and XA transactions. The InnobaseEngine is a plugin which inherits from plugin::XaStorageEngine because InnoDB supports SQL transactions as well as XA.

Explicit Statement and Transaction Boundaries

The second major change I made addressed the problem that Mark Callaghan noted in asking why finding out when a statement starts and ends was so obscure. I added two new methods to plugin::StorageEngine called doStartStatement() and doEndStatement(). The kernel now explicitly tells storage engines when a SQL statement starts and ends. This happens before any calls to Cursor::external_lock() happen, and there are no exception cases. In addition, the kernel now always tells transactional storage engines when a new SQL transaction is starting. It does this via an explicit call to plugin::TransactionalStorageEngine::doStartTransaction(). No exceptions, and yes, even for DDL operations.

What this means is that for a transactional storage engine, it no longer needs to “count the calls to Cursor::external_lock()” in order to know when a statement or transaction starts and ends. For a SQL transaction, this means that there is a clear code call path and there is no need for the storage engine to track whether the session is in AUTOCOMMIT mode or not. The kernel does all that work for the storage engine. Imagine a Session executes a single INSERT statement against an InnoDB table while in AUTOCOMMIT mode. This is what the call path looks like:

 drizzled::Statement::Insert::execute()
 |
 -> drizzled::mysql_lock_tables()
    |
    -> drizzled::TransactionServices::registerResourceForTransaction()
       |
       -> drizzled::plugin::TransactionalStorageEngine::startTransaction()
          |
          -> InnobaseEngine::doStartTransaction()
       |
       -> drizzled::plugin::StorageEngine::startStatement()
          |
          -> InnobaseEngine::doStartStatement()
       |
       -> drizzled::plugin::StorageEngine::getCursor()
          |
          -> drizzled::Cursor::write_row()
             |
             -> InnobaseCursor::write_row()
       |
       -> drizzled::TransactionServices::autocommitOrRollback()
          |
          -> drizzled::plugin::TransactionStorageEngine::commit()
             |
             -> InnobaseEngine::doCommit()

I think this will come as a welcome change to storage engine developers working with Drizzle.

No More Need for Engine to Call trans_register_ha()

There was an interesting comment in the original documentation for the transaction processing code. It read:

Roles and responsibilities
————————–

The server has no way to know that an engine participates in
the statement and a transaction has been started
in it unless the engine says so. Thus, in order to be
a part of a transaction, the engine must “register” itself.
This is done by invoking trans_register_ha() server call.
Normally the engine registers itself whenever handler::external_lock()
is called. trans_register_ha() can be invoked many times: if
an engine is already registered, the call does nothing.
In case autocommit is not set, the engine must register itself
twice — both in the statement list and in the normal transaction
list.

That comment, and I’ve read it dozens of times, always seemed strange to me. I mean, does the server really not know that an engine participates in a statement or transaction unless the engine tells it? Of course not.

So, I removed the need for a storage engine to “register itself” with the kernel. Now, the transaction manager inside the Drizzle kernel (implemented in the TransactionServices component) automatically monitors which engines are participating in an SQL transaction and the engine doesn’t need to do anything to register itself.

In addition, due to the break-up of the plugin::StorageEngine class and the XA API into plugin::XaResourceManager, Drizzle’s transaction manager can now coordinate XA transactions from plugins other than storage engines. Yep, that’s right. Any plugin which implements plugin::XaResourceManager can participate in an XA transaction and Drizzle will act as the transaction manager. What’s the first plugin that will do this? Drizzle’s transaction log. The transaction log isn’t a storage engine, but it is able to participate in an XA transaction, so it will implement plugin::XaResourceManager but not plugin::StorageEngine.

Performance Impact of Code Changes

So, that “yet another overhead” Monty talked about in the quote above? There wasn’t any noticeable impact in performance or scalability at all. So much for optimize-first coding.

What’s Next?

The next thing I’m working on is removing the notion of the “statement transaction”, which is also a historical by-product, this time because of BerkeleyDB. Gee, I’ve got a lot of work ahead of me…

^[1] Actually, there is a way that a transaction that was rolled back can get written to the transaction log. For bulk operations, the server can cut a Transaction message into multiple segments, and if the SQL transaction is rolled back, a special RollbackStatement message is written to the transaction log.

View full post on Planet Drizzle

What Is Linux :-

Linux is an operating system that was initially created as a
hobby by a young student, Linus Torvalds, at the University of
Helsinki in Finland. Linus had an interest in Minix, a small
UNIX system, and decided to develop a system that exceeded the
Minix standards.

Linux is developed under the GNU General Public License and its
source code is freely available to everyone. Linux may be used
for a wide variety of purposes including networking, software
development, and as an end-user platform.

Linux is often considered an excellent, low-cost alternative to
other more expensive operating systems. Due to the very nature
of Linux’s functionality and availability, it has become quite
popular worldwide and a vast number of software programmers have
taken Linux’s source code and adapted it to meet their
individual needs.

At this time, there are dozens of ongoing projects for porting
Linux to various hardware configurations and purposes. Here are
10 reasons why Linux could be the best operating system for you.

A Linux Distribution has thousands of dollars worth of software
for no cost (or a couple of dollars if purchased on CD). Linux
is a complete operating system that is :-

Stable – the crash of an application is much less likely to
bring down the operating system under Linux

Reliable – Linux servers are often up for hundreds of days
compared with the regular reboots required with a Windows
system. Extremely powerful.

Comes with a complete development environment, including C,
C++, Fortran compilers, and toolkits such as Qt and scripting
languages such as Perl, Awk and sed. A C compiler for Windows
alone would set you back hundreds of dollars.

Excellent networking facilities: allowing you to share CPUs,
share modems etc; all of which are not included or available
with Windows 95 The ideal environment to run servers such as a
web server (e.g. Apache), or an FTP server.

A wide variety of commercial software is available if the free
software doesn’t satisfy your needs.

An operating system that is easily upgradeable. After any
length of time a typical installation of Windows and software
gets into a complete mess. Often the only way to clear out all
the debris is to reformat the hard disk and start again. Linux,
however, is much better for maintaining the system.

Supports multiple processors as standard.

True multi-tasking; the ability to run more than one program at
the same time.

Excellent window systems called X; the equivalent of Windows
but much more flexible.

Linux holds a major share as a server operating system and has
the largest market share for the entire Internet. It is also
increasingly popular on the desktop as well. Internet Presence
Division provides value-added service and support on all
versions of Linux.

For more articles Please visit Articles at
All Web Hosts

Product Description
Quickly master dynamic, database-driven Web development—no experience necessary!Even if you’re completely new to PHP, MySQL, and Web database development, this book will guide you through every step of building powerful, database-driven, dynamic Web sites. Direct from the world’s top scripting instructor, Ellie Quigley, PHP and MySQL by Example illuminates every concept with tested code examples, screen shots showing program output, and clear, line-by-l… More >>

PHP and MySQL by Example

Are you unable to mount your Linux hard drive volume? Are you facing superblock, inode, or file system corruption like error messages while booting your Linux system? Such situations are caused by corruption of Linux hard drive metadata structures. This behavior of Linux operating system renders all your significant data inaccessible and leads to significant data loss. In order to get your mission-critical data recovered, you should opt for Linux Data Recovery solutions through reliable tools.

In a practical scenario, you may encounter the below error messages after copying your data through ddrescue command-line tool and running e2fsck utility:

“Superblock has a bad ext3 journal (inode 8).
Clear? Yes”

Or

“/dev/sdf was not cleanly unmounted, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Missing ‘..’ in directory inode 1785876.
Fix? Yes

Pass 3: Checking directory connectivity
Root inode is not a directory; aborting.
e2fsck: aborted
fsck.ext2 /dev/sdf failed (status 0×8). Run manually.”

After the above error message, you can not access your precious data from the hard drive. In order to get your lost data back, and to sort out these errors, you should perform Data Recovery Linux.

Resolution

The problem is caused by bad block at the starting of inode table. You can fix the problem by reconstructing root inode. It is quite easy to reconstruct the root inode, however the problem is that root inode needs to be placed at fixed location and currently a bad block is located there.

Forcing the hard drive to use spare block from its pool is possible, however a bad block is often an indicator of the beginning of hard drive crash. Thus, replacing the affected hard drive with new one is the safest way to recover from this situation. Use ‘dd’ command to copy data from old hard drive to new one. Then run e2fsck to create a new root directory on the new hard drive, and shift all the inodes, respectively.

If the above method does not solve your problem, Linux Recovery applications perform absolute recovery. They use high-end scanning techniques to methodically scan the entire hard drive and get all your precious data recovered. The Linux Data Recovery software come equipped with simple and rich graphical user interface to enable easy recovery.

Stellar Phoenix Linux Data Recovery is the most effective solution for all logical data loss scenarios. The software recovers Ext4, Ext3, Ext2, FAT12, FAT16, and FAT32 file system volumes. It is compatible with all major distributions of Linux operating system such as Red Hat, Debian, Fedora, SUSE, and Ubuntu.

Roma Nell’anno Mdcccxxxviii , Volume 3

When you are going to hire a hosting service to build your website for your small business then you need to know the basic things of web hosting. You are the best person to judge that what required features are and what about your budget.

Commonly, this choice to host your website depends upon the scripting language that is used in your website. If languages like PHP, MySQL, or Perl are used in your website, Linux server hosting is preferable. But if you are thinking for ASP Dot Net scripting language then your finest choice would the window hosting.

If you need some interactive facilities like chat or searchable database, Linux may not be the best choice. Also, a Linux-based web server is not fully acquiescent with Windows technologies. If you are using any Windows-centric technologies like Visual Basic, then Windows-based servers would be the required choice for you.

Advantages of Linux server hosting

1. Linux is an open source software product and thus it does not need the high licensing fees that other operating systems do; you can freely download and use it at no cost.

2. A Linux website can easily be converted to a Windows website without much hassle. Also, the website can easily be changed as the requirement of the user grows.

3. Linux web hosting is very economical as Linux is a free operating system. Usually, only the cost of distribution is borne by the host or owner.

4. When the scripting language like PHP, MySQL, or Perl etc. are required for your website then Linux server hosting is the most reliable and cost effective solution for you. Unless heavy load of scripting language, this will not be traceable.

5. There are many kinds of databases that run on Linux hosting, but the most widespread most likely among web host providers are mSQL, MySQL and PostgreSQL. These databases are relational by nature, and permit vastly optimized communication with your website for rapid reclamation of data.

6. By lots of circles Linux hosting is supposed to be much securing than Windows hosting, and for that reason only Linux server hosting is also the more popular choice for web designers and programmers.

Advantages of windows server hosting

1. .NET technologies like ASP dot net VB dot net are available on this platform. If you have a website built with Microsoft .NET technologies then you will have to choose a Windows server hosting plan.

2. If you need an enterprise class database there are some different features that run on a Linux platform but if you need Microsoft’s MSSQL database, a Windows hosting plan will be your best choice.

3. Access Databases are only obtainable with a Windows server hosting plan. Access can only run on a Microsoft Windows platform and thus it is unavailable on a Linux hosting. If your site needs particularly this feature then you will have to accept windows server hosting.

4. If you are using some of the Microsoft’s Sharepoint services a Windows Hosting plan is only reliable choice for you.

5. Windows plans typically much costly because of the license costs compulsory by Microsoft.

Normally, Linux server hosting plans are at least 20% economical than windows server hosting plans. If you need not to have any unique features for your website, like cart, searchable database etc, Linux server hosting is a finest alternative for your needs.

When you are required for specific Microsoft technologies, Windows hosting plans generally will be your favorable think. But for all small business owner needs Linux hosting plans will generally perform quite well for its cost effective features.