Oracle DBA|MySQL DBA|数据库管理，架构，监控与性能优化 --- NinGoo.net

记一次redo损坏导致ora-600[4000]的恢复

据说今天是光棍节，逢年过节，必有大事。快下班的时候，一位朋友碰到了一个大问题，数据库服务器异常断电重启以后，数据库无法启动，报ora-600[4000]错误，尝试了使用隐藏参数，还是无法打开。

ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00600: internal error code, arguments: [4000], [46], [], [], [], [], [], []

从ora-600内部错误的类型可以看到，4000是trasaction undo相关的内部错误。搜索一下可以发现，logzgh同学已经两次碰到过该问题了(第一次，第二次)。看来在异常宕机的情况下，这个问题出现的机率还是比较高的。既然是有先例的，恢复起来应该是没有问题的，这个案例和logzgh的第一个案例基本一样，trace文件里显示也是obj$上有异常事务:

ORA-00600: internal error code, arguments: [4000], [46], [], [], [], [], [], []
Current SQL statement for this session:
select ctime, mtime, stime from obj$ where obj# = :1

将trace文件往下翻，可以看到出现问题的block的dump信息，在block的itl中果然有一条活动事务存在。用bbed找到对应块，设置offset到61，将itl的flag从20改为80，重新计算block的checksum并apply。然后尝试重新启动数据库，这次ora-600[4000]错误没有了，报的是ora-600[2662]错误，数据库无法open。2662就比较容易了，网上相关的处理案例已经一大堆了，将数据库启动到mount状态，设置10015事件来调整scn即可

alter session set events '10015 trace name adjust_scn level 1';

如果数据库已经处于open状态，则可以使用如下语句来调整SCN:

alter session set events 'IMMEDIATE trace name ADJUST_SCN level x';

再次重新打开库，报ora-600[2256]，继续使用10015事件，这次level设置为(ora-600错误的第三个参数+1)*4

alter session set event '10015 trace name adjust_scn level 20';

重启，报ora-600[4097]错误，好了，现在基本上恢复已经接近尾声了，使用下面几个隐藏参数，可以正常将数据库打开了。接下来就是告诉朋友做全库export/重建库/import了。

_allow_resetlogs_corruption=true
_allow_terminal_recovery_corruption=true
_corrupted_rollback_segments=(_SYSSMU1$, _SYSSMU2$, _SYSSMU3$, _SYSSMU4$, _SYSSMU5$, _SYSSMU6$,
 _SYSSMU7$, _SYSSMU8$, _SYSSMU9$, _SYSSMU10$)

年关将近，事故高发时期，阿弥陀佛，多求个平安吧。话说这个案例还没来得及些写呢，晚上回家的路上又接到说有人truncate了一张表。。。

Posted by NinGoo at 2009-11-11 22:23 | 『数据库』 | 1 Comment »

性能测试工具sysbench简介

sysbench是一款开源的多线程性能测试工具，可以执行CPU/内存/线程/IO/数据库等方面的性能测试。数据库目前支持MySQL/Oracle/PostgreSQL。本文只是简单演示一下几种测试的用法，后续准备利用sysbench来对MySQL进行一系列的测试。具体的一些参数设置，需要根据不同的测试要求来进行调整。

下载
http://sourceforge.net/projects/sysbench/

编译安装
默认支持MySQL，如果需要测试Oracle/PostgreSQL，则在configure时需要加上–with-oracle或者–with-pgsql参数

./configure --prefix=/u01/sysbench \
--with-mysql-includes=/opt/mysql/include/mysql \
--with-mysql-libs=/opt/mysql/lib/mysql

make && make install

参数

NinGoo:/u01/sysbench/bin>$sysbench
Missing required command argument.
Usage:
  sysbench [general-options]... --test= [test-options]... command

General options:
  --num-threads=N            number of threads to use [1]
  --max-requests=N           limit for total number of requests [10000]
  --max-time=N               limit for total execution time in seconds [0]
  --forced-shutdown=STRING   amount of time to wait after --max-time before forcing shutdown [off]
  --thread-stack-size=SIZE   size of stack per thread [32K]
  --init-rng=[on|off]        initialize random number generator [off]
  --test=STRING              test to run
  --debug=[on|off]           print more debugging info [off]
  --validate=[on|off]        perform validation checks where possible [off]
  --help=[on|off]            print help and exit
  --version=[on|off]         print version and exit

Compiled-in tests:
  fileio - File I/O test
  cpu - CPU performance test
  memory - Memory functions speed test
  threads - Threads subsystem performance test
  mutex - Mutex performance test
  oltp - OLTP test

Commands: prepare run cleanup help version
See 'sysbench --test= help' for a list of options for each test.

[继续阅读全文]

Posted by NinGoo at 2009-11-08 12:14 | 『工具』 | 4 Comments »

InnoDB线程并发检查机制

InnoDB在接受MySQL线程调用能时，有一个并发线程的检查机制，通过innodb_thread_concurrency参数进行控制。如果参数设置大于0，则表示检查机制开启，允许进入的线程数就是参数的值。等于0则禁用并发检查。

在新的MySQL线程调用Innodb接口前，Innodb会检查已经接受的请求线程数，如已经超过innodb_thread_concurrency设置的限制，则该请求线程会等待innodb_thread_sleep_delay微秒后尝试重新请求，如果第二次请求还是无法获得，则该线程会进入线程队列休眠。重试两次的机制是为了减少CPU的上下文切换的次数，以降低CPU消耗，这和Oracle中latch的spin机制是同样的道理。如果请求被Innodb接受，则会获得一个次数为innodb_concurrency_tickets(默认500次)的通行证，在次数用完之前，该线程重新请求时无须再进行前面所说innodb_thread_concurrency的检查。

上述检查逻辑在源码storage/innobase/srv/srv0srv.c(Innodb很多参数都可以在该文件中找到定义)的srv_conc_enter_innodb函数中，有兴趣的可以仔细阅读一下，代码比较浅显，不难理解。另外，如果是一个已经持有lock的线程，则通过调用srv_conc_force_enter_innodb函数可以无视该检查，这是为了避免线程长时间持有锁影响性能，且可能增加死锁的机率。除此之外，slave线程也是有无视检查直接通行的权限。

简单思考一下上述机制，可以得出一个初步的推论：在数据库并发请求较小的情况下，从性能上来说禁用检查机制应该是更好的，毕竟执行检查机制本身也需要加锁(Mutex)。当并发线程很高的情况下，则开启检查机制对性能更有利。至于具体innodb_thread_concurrency设置为多少，可能就需要在不同的条件下实际的做一下测试了，不同的硬件环境，不同的MySQL版本和Innodb版本，应该都会有一些区别。

源代码中对于innodb_thread_concurrency参数的注释如下：

/* The following controls how many threads we let inside InnoDB concurrently:
threads waiting for locks are not counted into the number because otherwise
we could get a deadlock. MySQL creates a thread for each user session, and
semaphore contention and convoy problems can occur withput this restriction.
Value 10 should be good if there are less than 4 processors + 4 disks in the
computer. Bigger computers need bigger values. Value 0 will disable the
concurrency check. */

ulong   srv_thread_concurrency  = 0;

因为检查机制需要Mutex保护(Mutex-based Model)，所以开启检查本身也有性能消耗，并且扩展性也会受到限制，在MySQL5.4版本中引入了一种新的机制（Timer-based Model），这里就不讨论了，有兴趣的可以参考这里(需要翻墙)，实际上XtraDB存储引擎里已经包含Timer-based Model，通过参数innodb_thread_concurrency_timer_based可以开启，默认为OFF。在MySQL5.4的srv0srv.c的源代码中的注释中，可以看到Google和Percona的版权声明，看来MySQL5.4中吸引了很多第三方的改进代码，值得期待。

Posted by NinGoo at 2009-11-07 21:52 | 『数据库』 | No Comments »

从show innodb status看XtraDB的增强特性

XtraDB存储引擎相对于标准的InnoDB引擎做了很多的改进，从show innodb status的结果可以初步看到部分增强的特性。下面是一个XtraDB测试环境(Linux)的结果：

root@(none) 11:25:18>show innodb status\G
*************************** 1. row ***************************
  Type: InnoDB
  Name:
Status:
=====================================
091106 11:26:11 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 53 seconds
----------
BACKGROUND THREAD
----------
srv_master_thread loops: 8 1_second, 8 sleeps, 0 10_second, 15 background, 15 flush
srv_master_thread log flush and writes: 8  log writes only: 39
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 9, signal count 9
Mutex spin waits 2, rounds 60, OS waits 0
RW-shared spins 9, OS waits 9; RW-excl spins 0, OS waits 0
Spin rounds per wait: 30.00 mutex, 30.00 RW-shared, 0.00 RW-excl
--------
FILE I/O
--------
I/O thread 0 state: waiting for i/o request (insert buffer thread)
I/O thread 1 state: waiting for i/o request (log thread)
I/O thread 2 state: waiting for i/o request (read thread)
I/O thread 3 state: waiting for i/o request (read thread)
I/O thread 4 state: waiting for i/o request (read thread)
I/O thread 5 state: waiting for i/o request (read thread)
I/O thread 6 state: waiting for i/o request (write thread)
I/O thread 7 state: waiting for i/o request (write thread)
I/O thread 8 state: waiting for i/o request (write thread)
I/O thread 9 state: waiting for i/o request (write thread)
Pending normal aio reads: 0, aio writes: 0,
 ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0
Pending flushes (fsync) log: 0; buffer pool: 0
25 OS file reads, 48 OS file writes, 32 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 0.09 writes/s, 0.08 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 0, seg size 2,
0 inserts, 0 merged recs, 0 merges
Hash table size 553193, node heap has 1 buffer(s)
0.00 hash searches/s, 0.02 non-hash searches/s
---
LOG
---
Log sequence number 48613
Log flushed up to   48613
Last checkpoint at  48613
Max checkpoint age  7782360
Modified age        0
Checkpoint age      0
0 pending log writes, 0 pending chkp writes
24 log i/o's done, 0.04 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total memory allocated 136806400; in additional pool allocated 0
Internal hash tables (constant factor + variable factor)
    Adaptive hash index 2233316         (2212772 + 20544)
    Page hash           69556
    Dictionary cache    576800  (553912 + 22888)
    File system         41904   (41336 + 568)
    Lock system         167076  (166436 + 640)
    Recovery system     0       (0 + 0)
    Threads             41524   (41348 + 176)
Dictionary memory allocated 22888
Buffer pool size        8191
Buffer pool size, bytes 134201344
Free buffers            8170
Database pages          20
Modified db pages       0
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages read 19, created 2, written 45
0.00 reads/s, 0.00 creates/s, 0.04 writes/s
Buffer pool hit rate 1000 / 1000
LRU len: 20, unzip_LRU len: 0
I/O sum[2]:cur[0], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
1 read views open inside InnoDB
Main thread process no. 16905, id 2788809632, state: waiting for server activity
Number of rows inserted 4, updated 0, deleted 0, read 0
0.02 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
------------
TRANSACTIONS
------------
Trx id counter 509
Purge done for trx's n:o < 506 undo n:o < 0
History list length 1
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 0, not started, process no 16905, OS thread id 2777967520
MySQL thread id 6, query id 196 dxmonitor 172.19.68.250 root
show innodb status
---TRANSACTION 508, ACTIVE 2 sec, process no 16905, OS thread id 2777766816
1 lock struct(s), heap size 320, 0 row lock(s), undo log entries 1
MySQL thread id 7, query id 195 dxmonitor 172.19.68.250 root
TABLE LOCK table `test`.`test` trx id 508 lock mode IX
----------------------------
END OF INNODB MONITOR OUTPUT
============================

1 row in set, 1 warning (0.00 sec)

从上面的结果可以看到：

关于XtraDB的show innodb status的更多改进，请参考这里。

Posted by NinGoo at 2009-11-06 12:03 | 『数据库』 | 1 Comment »

Page 6 of 101123456789101112131415...Last »


常用标签： oracle life MySQL Oracle11g blog dba 新特性 oow2009 oow wordpress

最新评论 | Recent comments