The annoying thing about .ibd files you can’t easily copy the an .ibd file to another MySQL server. If you try to very often you’ll get an error in the log:

InnoDB: Error: tablespace id is 10 in the data dictionary
InnoDB: but in file ./sakila/actor.ibd it is 15!

However sometimes you have to connect the .ibd file to an alien ibdata1.

There are several situation when you have to:

1. ibdata1 is erroneously removed

2. ibdata1 is heavily corrupted and innodb_force_recovery doesn’t help

Chris Calender suggests two methods. The first is create/drop the table many times until space_id in InnoDB dictionary and .ibd file match. The second is to edit space_id inside .ibd file with a hex editor.

I would like to elaborate the second method.

But let’s understand first what’s going on and why InnoDB refuses to use suggested .ibd file.

There is an InnoDB dictionary. It consists of several internal tables. For our topic only SYS_TABLES and SYS_INDEXES are relevant. These are usual InnoDB tables, but they’re hidden from a user(you can see them in information_scheme database in Percona Server though).

The structure of these tables is following:

SYS_TABLES:

CREATE TABLE `SYS_TABLES` (
`NAME` varchar(255) NOT NULL default '',
`ID` bigint(20) unsigned NOT NULL default '0',
`N_COLS` int(10) default NULL,
`TYPE` int(10) unsigned default NULL,
`MIX_ID` bigint(20) unsigned default NULL,
`MIX_LEN` int(10) unsigned default NULL,
`CLUSTER_NAME` varchar(255) default NULL,
`SPACE` int(10) unsigned default NULL,
PRIMARY KEY  (`NAME`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1

SYS_INDEXES:

CREATE TABLE `SYS_INDEXES` (
`TABLE_ID` bigint(20) unsigned NOT NULL default '0',
`ID` bigint(20) unsigned NOT NULL default '0',
`NAME` varchar(120) default NULL,
`N_FIELDS` int(10) unsigned default NULL,
`TYPE` int(10) unsigned default NULL,
`SPACE` int(10) unsigned default NULL,
`PAGE_NO` int(10) unsigned default NULL,
PRIMARY KEY  (`TABLE_ID`,`ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1

Please note field SPACE. For table actor it is equal to 15:

mysql> select * from `INNODB_SYS_TABLES` where `SCHEMA` = 'sakila' AND `NAME` = 'actor'\G
*************************** 1. row ***************************
SCHEMA: sakila
NAME: actor
ID: 13
N_COLS: 2147483652
TYPE: 1
MIX_ID: 0
MIX_LEN: 0
CLUSTER_NAME:
SPACE: 15
1 row in set (0.00 sec)

SPACE is equal to 15 in all actor’s indexes:

mysql> select * from INNODB_SYS_INDEXES WHERE TABLE_ID = 13\G
*************************** 1. row ***************************
TABLE_ID: 13
ID: 15
NAME: PRIMARY
N_FIELDS: 1
TYPE: 3
SPACE: 15
PAGE_NO: 3
*************************** 2. row ***************************
TABLE_ID: 13
ID: 16
NAME: idx_actor_last_name
N_FIELDS: 1
TYPE: 0
SPACE: 15
PAGE_NO: 4
2 rows in set (0.00 sec)

In InnoDB world actor.ibd is a tablespace. It has space_id and it is equal to 15 for this particular table at this particular server.

As you can see secondary indexes are stored in actor.ibd as well.

But where is space_id in actor.ibd?

Like any other tablespace actor.ibd costsists of a set of InnoDB pages. A page is 16k long (UNIV_PAGE_SIZE in the source code).

Let’s take a look at the page header:

So, space_id is 4 bytes written to every InnoDB page. An .ibd file can be huge, while ibdata1 is usually smaller. Thus, it is easier to modify space_id in InnoDB dictionary once than in every InnoDB page.

How let’s connect actor.ibd from sakila database taken from some MySQL server.

0. Create empty InnoDB tablespace.

1. Create the table:

mysql>CREATE TABLE actor (
actor_id SMALLINT UNSIGNED NOT NULL AUTO_INCREMENT,
first_name VARCHAR(45) NOT NULL,
last_name VARCHAR(45) NOT NULL,
last_update TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY  (actor_id),
KEY idx_actor_last_name (last_name)
)ENGINE=InnoDB DEFAULT CHARSET=utf8;

This command will create respective records in SYS_TABLES and SYS_INDEXES.

2. Now let’s modify SPACE in InnoDB dictionary. MySQL must be stopped at this point. There is a tool ibdconnect in Percona InnoDB Recovery Tool. Make sure you’re using the latest version from the trunk.

It reads space_id from an .ibd file and updates the dictionary in ibdata1.

# ./ibdconnect -o /var/lib/mysql/ibdata1 -f /var/lib/mysql/sakila/actor.ibd -d sakila -t actor
actor.ibd
actor.ibd belongs to space #15
... Skipped output...
SYS_TABLES is updated successfully
... Skipped output...
SYS_INDEXES is updated successfully

It is possible that space_id from actor.ibd is already used by some other table.
In this case if ibdata was updated MySQL will fail to start with error:

InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Error: trying to add tablespace 15 of name './sakila/customer.ibd'
InnoDB: to the tablespace memory cache, but tablespace
InnoDB: 15 of name './sakila/actor.ibd' already exists in the tablespace
InnoDB: memory cache!

To refrain from such error ibdconnect does check if the space_id is already used.
It will refuse to update ibdata1:

$ ./ibdconnect -o ibdata1 -f t2.ibd -d sakila -t actor
...
Error: Space id: 12 is already used in InnoDB dictionary for table test/t2

In this case you need to drop table `test`.`t2` and create it again. InnoDB will assign other space_id, thus 12 will be freed.

3. Now SPACE is modified in the dictionary, but checksums are bad. To regenerate them use innochecksum from the same toolset. Run it two times:

# ./innochecksum -f /var/lib/mysql/ibdata1
page 8 invalid (fails new style checksum)
page 8: new style: calculated = 0x B7C5C82C; recorded = 0x BFE71C21
fixing new checksum of page 8
page 11 invalid (fails new style checksum)
page 11: new style: calculated = 0x E4189B9B; recorded = 0x C168689B
fixing new checksum of page 11
#
# ./innochecksum -f /var/lib/mysql/ibdata1
page 8 invalid (fails old style checksum)
page 8: old style: calculated = 0x 8195646B; recorded = 0x DA79A2EE
fixing old checksum of page 8
page 8 invalid (fails new style checksum)
page 8: new style: calculated = 0x 119FD630; recorded = 0x B7C5C82C
fixing new checksum of page 8
page 11 invalid (fails old style checksum)
page 11: old style: calculated = 0x 908297E7; recorded = 0x 6536CEE8
fixing old checksum of page 11
page 11 invalid (fails new style checksum)
page 11: new style: calculated = 0x D5DC3269; recorded = 0x E4189B9B
fixing new checksum of page 11

4. The third time to be sure ibdata1 has valid checksums:

#./innochecksum /var/lib/mysql/ibdata1
#

5. Now you can start MySQL and take a dump from the table. The table is accessible, but due to obvious reason it shold not be used in production.

The tool ibdconnect was tested on MySQL 5.1 on CentOS 5.6 x86_64. However it expected to work on all versions of MySQL/InnoDB.

View full post on MySQL Performance Blog

please give me links and/or tips.

i really need to make this in 3 days. :’c

thank you.

Product Description
Today’s system administrators deal with a vast number of situations, operating systems, software packages, and problems. Those who are in the know have kept their copy of Linux Server Hacks close at hand to ease their burden. And while this helps, it’s not enough: any sys admin knows there are many more hacks, cool tips, and ways of solving problems than can fit in a single volume (one that mere mortals can lift, that is). Which is why we created Linux Server Hacks… More >>

Linux Server Hacks, Volume Two: Tips & Tools for Connecting, Monitoring, and Troubleshooting

I’ll demonstrate a simple example on how to connect to mysql using PHP’s PDO class. Just some of the benefits of PDO is that it’s fast and if you use the PDO::prepare() method it will prevent SQL injection attacks by calling the PDO::quote() method. The other pros is that there are several databases it will support. So let’s dive right into the code:

$hostname = ‘localhost’;
$username = ‘your_username’;
$password = ‘your_password’;

try {
$db = new PDO(“mysql:host=$hostname;dbname=mysql”, $username, $password);
echo ‘Connected to database’;
}
catch(PDOException $e) {
echo $e->getMessage();
}

Fatal Error new PDO Instance

Just an important note that if you receive the following type of fatal error in your development environment:

Fatal error: Uncaught exception ‘PDOException’ with message ‘SQLSTATE[42000] [1049] Unknown database ”user”’ in C:Program FilesApache Software FoundationApache2.2htdocstesttrunkcodelogin1classesstd.pdo_singleton.class.inc:30 Stack trace: #0 C:Program FilesApache Software FoundationApache2.2htdocstesttrunkcodelogin1classesstd.pdo_singleton.class.inc(30): PDO->__construct(‘mysql:host=loca…’, ‘username’, ‘password’) #1 C:Program FilesApache Software FoundationApache2.2htdocstesttrunkcodelogin1classesstd.mysql.class_test.inc(43): db::getConnect() #2 C:Program FilesApache Software FoundationApache2.2htdocstesttrunkcodelogin1connect.php(6): MySqlDb->confirmUserPass(‘usertest’, ‘passtest’) #3 {main} thrown in C:Program FilesApache Software FoundationApache2.2htdocstesttrunkcodelogin1classesstd.pdo_singleton.class.inc on line 30

Looks pretty messy and hard to decipher. When trying to decipher error code I usually look at the first error which led me to see why it was reporting an “Unknown database” when it was existing. The extra quotes also gave me a hint as to the problem. So I concluded the problem resulted in the placement of extra quotes around the values of host and/or dbname. The following will generate the previous error:

$db = new PDO(“mysql:host=’localhost’;dbname=’mysql’”, $username, $password);

So if you do not use variables then do not add the single quotes for the values of host and dbname. In other words, use the following code instead:

$db = new PDO(“mysql:host=localhost;dbname=mysql”, $username, $password);

This is a simple test to connect to your mysql database or any other support database. Just for your information if you wish to connect to PostgreSQL which is another popular and robust database use the following code in place of the line of instantiation:

$db = new PDO(“pgsql:dbname=pdo;host=localhost”, “username”, “password” );

If you’re in a development environment and wish to display your errors directly to the screen you can specify the errors displayed. You need to set your display_errors settings to ‘on’ in your php.ini file. So set your error attributes after instantiating the PDO class like so:

$db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);

There are three types of error report settings for PDO::ATTR_ERRMODE:
PDO::ERRMODE_SILENT = error codes
PDO::ERRMODE_WARNING = E_WARNING
PDO::ERRMODE_EXCEPTION = Throw exceptions

Here’s an example of implementing PDO::ATR_ERRMODE:

$hostname = ‘localhost’;
$username = ‘your_username’;
$password = ‘your_password’;

try {
$db = new PDO(“mysql:host=$hostname;dbname=articles”, $username, $password);

echo ‘Connected to database’; // check for connection

$db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_WARNING);

$sql = ‘Select * from tutorialref where id=1′;
$result = $db->query($sql);
foreach ($result as $row) {
echo $row['id'] .’ – ‘. $row['author'] . ‘
‘;
}

$db = null; // close the database connection

}
catch(PDOException $e) {
echo $e->getMessage();
}

Don’t confuse this with the errors generated from the php setting of error_reporting. The errors from PDO::ATTR_ERRMODE apply to the sql query and its results. I’ll dive into the error settings and the different outputs of the php.ini error_reporting settings and PDO::ATTR_ERRMODE report settings in a future article.