Weez.com » optimize http://www.weez.com Solving everyday practical LAMP problems... one at a time Sat, 11 Feb 2012 03:24:53 +0000 en hourly 1 http://wordpress.org/?v=3.0.1 Thinking about running OPTIMIZE on your Innodb Table ? Stop! http://www.weez.com/2010/12/thinking-about-running-optimize-on-your-innodb-table-stop/ http://www.weez.com/2010/12/thinking-about-running-optimize-on-your-innodb-table-stop/#comments Thu, 09 Dec 2010 21:46:03 +0000 Abidoon http://www.weez.com/2010/12/thinking-about-running-optimize-on-your-innodb-table-stop/ Innodb/XtraDB tables do benefit from being reorganized often. You can get data physically laid out in primary key order as well as get better feel for primary key and index pages and so using less space,
it is just OPTIMIZE TABLE might not be best way to do it.

If you’re running Innodb Plugin on Percona Server with XtraDB you get benefit of a great new feature – ability to build indexes by sort instead of via insertion. This process can be a lot faster, especially for large indexes which would get inserts in very random order, such as indexes on UUID column or something similar. It also produces a lot better fill factor. The problem is…. OPTIMIZE TABLE for Innodb tables does not get advantage of it for whatever reason.

Lets take a look at little benchmark I done by running OPTIMIZE for a second time on a table which is some 10 times larger than amount of memory I allocated for buffer pool:

PLAIN TEXT
SQL:

  1. CREATE TABLE `a` (
  2.   `id` int(10) UNSIGNED NOT NULL AUTO_INCREMENT,
  3.   `c` char(64) DEFAULT NULL,
  4.   PRIMARY KEY (`id`),
  5.   KEY `c` (`c`)
  6. ) ENGINE=InnoDB AUTO_INCREMENT=12582913 DEFAULT CHARSET=latin1
  7.  
  8. mysql> SELECT * FROM a ORDER BY id LIMIT 10;
  9. +—-+——————————————+
  10. | id | c                                        |
  11. +—-+——————————————+
  12. |  1 | 813cf02d7d65de2639014dd1fb574d4c481ecac7 |
  13. |  2 | 62960f5d5d50651e5a5983dacaedfa9a73a9ee87 |
  14. |  3 | cea33998792ffe28b16b9272b950102a9633439f |
  15. |  4 | 8346a7afa0a0791693338d96a07a944874340a1c |
  16. |  5 | b00faaa432f507a0d16d2940ca8ec36699f141c8 |
  17. |  6 | 8e00926cf6c9b13dc8e0664a744b7116c5c61036 |
  18. |  7 | f151fe34b66fd4d28521d5e7ccb68b0d5d81f21b |
  19. |  8 | 7fceb5afa200a27b81cab45f94903ce04d6f24db |
  20. |  9 | 0397562dc35b5242842d68de424aa9f0b409d60f |
  21. | 10 | af8efbaef7010a1a3bfdff6609e5c233c897e1d5 |
  22. +—-+——————————————+
  23. 10 rows IN SET (0.04 sec)
  24.  
  25. # This is just random SHA(1) hashes
  26.  
  27. mysql> OPTIMIZE TABLE a;
  28. +——–+———-+———-+——————————————————————-+
  29. | TABLE  | Op       | Msg_type | Msg_text                                                          |
  30. +——–+———-+———-+——————————————————————-+
  31. | test.a | OPTIMIZE | note     | TABLE does NOT support OPTIMIZE, doing recreate + analyze instead |
  32. | test.a | OPTIMIZE | STATUS   | OK                                                                |
  33. +——–+———-+———-+——————————————————————-+
  34. 2 rows IN SET (3 hours 3 min 35.15 sec)
  35.  
  36. mysql> ALTER TABLE a DROP KEY c;
  37. Query OK, 0 rows affected (0.46 sec)
  38. Records: 0  Duplicates: 0  Warnings: 0
  39.  
  40. mysql> OPTIMIZE TABLE a;
  41. +——–+———-+———-+——————————————————————-+
  42. | TABLE  | Op       | Msg_type | Msg_text                                                          |
  43. +——–+———-+———-+——————————————————————-+
  44. | test.a | OPTIMIZE | note     | TABLE does NOT support OPTIMIZE, doing recreate + analyze instead |
  45. | test.a | OPTIMIZE | STATUS   | OK                                                                |
  46. +——–+———-+———-+——————————————————————-+
  47. 2 rows IN SET (4 min 5.52 sec)
  48.  
  49. mysql> ALTER TABLE a ADD KEY(c);
  50. Query OK, 0 rows affected (5 min 51.83 sec)
  51. Records: 0  Duplicates: 0  Warnings: 0

That’s right ! Optimizing table straight away takes over 3 hours, while dropping indexes besides primary key, optimizing table and adding them back takes about 10 minutes, which is close than 20x speed difference and more compact index in the end.

So if you’re considering running OPTIMIZE on your tables consider using this trick, it is especially handy when you’re running it on the Slave where it is OK table is exposed without indexes for some time.
Note though nothing stops you from using LOCK TABLES on Innodb table to ensure there is not ton of queries starting reading table with no indexes and bringing box down.

You can also use this trick for ALTER TABLE which requires table rebuild. Dropping all indexes; doing ALTER and when adding them back can be a lot faster than straight ALTER TABLE.

P.S I do not know why this was not done when support for creating index by sorting was implemented. It looks very strange to me to have this feature implemented but majority of high level commands
or tools (like mysqldump) do not get advantage of it and will use old slow method of building indexes by insertion.

Entry posted by peter |
3 comments

Add to: delicious | digg | reddit | netscape | Google Bookmarks

View full post on MySQL Performance Blog

]]> http://www.weez.com/2010/12/thinking-about-running-optimize-on-your-innodb-table-stop/feed/ 3 SapientNitro Selects KENSHOO as Technology of Choice to Optimize Value of Search Engine Marketing for their Clients http://www.weez.com/2010/07/sapientnitro-selects-kenshoo-as-technology-of-choice-to-optimize-value-of-search-engine-marketing-for-their-clients/ http://www.weez.com/2010/07/sapientnitro-selects-kenshoo-as-technology-of-choice-to-optimize-value-of-search-engine-marketing-for-their-clients/#comments Mon, 26 Jul 2010 02:18:05 +0000 Abidoon http://www.weez.com/2010/07/sapientnitro-selects-kenshoo-as-technology-of-choice-to-optimize-value-of-search-engine-marketing-for-their-clients/ SapientNitro Selects KENSHOO as Technology of Choice to Optimize Value of Search Engine Marketing for their Clients
KENSHOO helps boost ROI of online ad spend for high profile international brands, including Hugo Boss and TracFone

Read more on PRWeb via Yahoo! News

]]> http://www.weez.com/2010/07/sapientnitro-selects-kenshoo-as-technology-of-choice-to-optimize-value-of-search-engine-marketing-for-their-clients/feed/ 0 MySQL Optimizations on Linux http://www.weez.com/2008/09/mysql-optimizations-on-linux/ http://www.weez.com/2008/09/mysql-optimizations-on-linux/#comments Fri, 12 Sep 2008 07:04:58 +0000 Abidoon http://www.weez.com/?p=22 It is important to optimize all pieces of the LAMP acronym and MySQL is not exception to the rule when deploying on the live servers

Here be the compile time flags.

export CFLAGS=”-O3 -march=cpu -pipe -msse2 -m3dnow -fomit-frame-pointer -mtune=cpu” CXX=gcc CXXFLAGS=”-O3 -cpu -felide-constructors -fno-exceptions -fno-rtti”

As mentioned in my previous entry you can go check the GCC Manual pages to replace the cpu keyword in the mtune and march in the CFLAGS and the -cpu in the CXXFLAGS.

I got approximately a 5% performance boost with these during benchmarking.

Please note though that the developer team got a much bigger increase in performance when they optimized the tables by adding indexes and improving their queries.

In general we also found that a simple select query took about twice as much time when the table was using InnoDB compared to MyISAM.

I am open to any additional feedback and ideas anyone may have.

]]> http://www.weez.com/2008/09/mysql-optimizations-on-linux/feed/ 0 PHP Optimizations http://www.weez.com/2008/09/php-optimizations/ http://www.weez.com/2008/09/php-optimizations/#comments Fri, 12 Sep 2008 00:11:01 +0000 Abidoon http://www.weez.com/?p=3 Anyone ever try and optimize PHP recently?

The following settings worked really well for me

1 – export CFLAGS=”-march=cpu -O3 -pipe -msse2 -m3dnow -fomit-frame-pointer -mtune=cpu”

Based on whatever processor architecture you were using you should then replace the cpu in the above line with the name of the processor as described in GCC Manual

2 – Try configuring with the following flags added: –enable-inline-optimization

3 – Once compiled use strip to reduce the PHP binary size like so -> strip php-cgi :)

4 – For a significant performance boost I would recommend the installation of an opcode cache such as x-cache. If you don’t know what an opcode cacher is or does go here

The above mentioned optimizations led to more than 50% increase in speed and a more than 50% drop in cpu usage. Please feel free to make additional suggestions on how I can squeeze more performance  out :) !

Cheers

]]> http://www.weez.com/2008/09/php-optimizations/feed/ 0