jbd2 mysql_jbd2 BUG · xiewen/xiewen.github.io Wiki · GitHub
現象：MySQl 掛起，不能寫入數據，或者整個系統(tǒng)崩潰掉
1. BUG 1 - MySQL 掛起
1.1. 癥狀
kernel: ------------[ cut here ]------------
kernel: WARNING: at fs/jbd2/journal.c:507 __jbd2_log_start_commit+0x72/0xf0 [jbd2]()
kernel: jbd: bad log_start_commit: 2768618130 2768618130 49048062 2768618131
kernel: Modules linked in: nfs lockd fscache auth_rpcgss nfs_acl sunrpc ip_vs_wlc ip_vs nf_conntrack ipv6 libcrc32c ovmapi(U) xen_netfront pcspkr ext4 mbcache jbd2 xen_blkfront dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
kernel: Pid: 30273, comm: mysqld Not tainted 2.6.39-200.32.1.el6uek.x86_64 #1
kernel: Call Trace:
kernel: [] warn_slowpath_common+0x7f/0xc0
kernel: [] warn_slowpath_fmt+0x46/0x50
kernel: [] __jbd2_log_start_commit+0x72/0xf0 [jbd2]
kernel: [] jbd2_log_start_commit+0x30/0x50 [jbd2]
kernel: [] ext4_sync_file+0x162/0x2d0 [ext4]
kernel: [] vfs_fsync_range+0x7f/0xa0
kernel: [] vfs_fsync+0x1c/0x20
kernel: [] do_fsync+0x3a/0x60
kernel: [] sys_fsync+0x10/0x20
kernel: [] system_call_fastpath+0x16/0x1b
kernel: ---[ end trace 75c73bd1e52b1c73 ]---
kernel: jbd2_log_wait_commit: error: j_commit_request=-1526349166, tid=49048062
# grep each /proc/fs/jbd2/*/info
/proc/fs/jbd2/dm-0-8/info:2981802046 transaction, each up to 8192 blocks
/proc/fs/jbd2/dm-1-8/info:5620 transaction, each up to 8192 blocks
1.2. BUG 描述
ext4/jbd2: don't wait (forever) for stale tid caused by wraparound (uek3, kernel.org)
1.3. rhel/ol 5 修復版本
2.6.18-371.12.1.el5 或 el/ol 5.11(2.6.18-398.el5) (1)
[fs] jbd: don't wake kjournald unnecessarily (Denys Vlasenko) [1116027
1081785]
[fs] jbd: don't wait (forever) for stale tid caused by wraparound
(Denys Vlasenko) [1116027 1081785]
[fs] ext4: fix waiting and sending of barrier in ext4_sync_file()
(Denys Vlasenko) [1116027 1081785]
[fs] jbd2: Add function jbd2_trans_will_send_data_barrier() (Denys
Vlasenko) [1116027 1081785]
[fs] jbd2: fix sending of data flush on journal commit (Denys
Vlasenko) [1116027 1081785]
[fs] ext4, jbd2: Add barriers for file systems with ext journals
(Denys Vlasenko) [1116027 1081785]
[fs] jbd: fix fsync() tid wraparound bug (Denys Vlasenko) [1116027
1081785] | 2.6.32-358.18.1.el6 或 6.5(2.6.32-431.el6) (ol)
[fs] ext4/jbd2: dont wait (forever) for stale tid caused by wraparound
(Eric Sandeen) [963557 955807]
[fs] jbd: dont wait (forever) for stale tid caused by wraparound (Eric
Sandeen) [963557 955807]
1.4. rhel/ol 6 修復版本
2.6.32-358.18.1.el6 或 6.5(2.6.32-431.el6) (ol)
[fs] ext4/jbd2: dont wait (forever) for stale tid caused by wraparound
(Eric Sandeen) [963557 955807]
[fs] jbd: dont wait (forever) for stale tid caused by wraparound (Eric
Sandeen) [963557 955807]
1.5. ol uek 修復版本
無
1.6. ol uek 3 修復版本
3.8.13-16 (1)
2. BUG 2 - 系統(tǒng)崩潰
2.1. 癥狀
# ----------- [cut here ] --------- [please bite here ] ---------
Kernel BUG at fs/jbd2/commit.c:395
invalid opcode: 0000 [1] SMP
last sysfs file: /devices/system/cpu/cpu0/topology/physical_package_id
CPU 0
Modules linked in: autofs4 i2c_dev i2c_core hidp nfs nfs_acl rfcomm l2cap bluetooth lockd sunrpc ip6t_REJECT xt_tcpudp ip6table_filter ip6_tables x_tables ipv6 xfrm_nalgo crypto_api ext4 jbd2 crc16 dm_mirror parport_pc lp parport xennet pcspkr xenblk dm_raid45 dm_message dm_region_hash dm_log dm_mod dm_mem_cache ata_piix libata scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd
Pid: 986, comm: jbd2/dm-0-8 Not tainted 2.6.18-274.0.0.0.1.el5xen #1
RIP: e030:[] [] :jbd2:jbd2_journal_commit_transaction+0x50/0x1074
RSP: e02b:ffff8803fb97bd70 EFLAGS: 00010246
RAX: 0000000000000008 RBX: ffff8803fb968800 RCX: ffff8803fb97be80
RDX: 0000000000000008 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffff8803fb968800 R08: ffff8803fb968898 R09: 0000000000000000
R10: ffff8803e8a6a488 R11: 0000000000000000 R12: ffff8803fb968824
R13: ffff8803fb9688c0 R14: 0000000000000000 R15: ffff8803fb968890
FS: 00002aaaaaac5330(0000) GS:ffffffff8062e000(0000) knlGS:0000000000000000
CS: e033 DS: 0000 ES: 0000
Process jbd2/dm-0-8 (pid: 986, threadinfo ffff8803fb97a000, task ffff8803ffafa080)
Stack: ffff8803fc2d1000 00000f7c00000000 0000000800000000 00000001ffffffff
ffff880300000001 00000000eb6ca2da 0000000000000000 0000000000000000
0000000000000000 0000000000000000
Call Trace:
[] _spin_lock_irqsave+0x9/0x14
[] lock_timer_base+0x1b/0x3c
[] try_to_del_timer_sync+0x7f/0x88
[] :jbd2:kjournald2+0x9a/0x1ec
[] autoremove_wake_function+0x0/0x2e
[] keventd_create_kthread+0x0/0xc4
[] :jbd2:kjournald2+0x0/0x1ec
[] keventd_create_kthread+0x0/0xc4
[] kthread+0xfe/0x132
[] child_rip+0xa/0x12
[] keventd_create_kthread+0x0/0xc4
[] kthread+0x0/0x132
[] child_rip+0x0/0x12
Code: 0f 0b 68 1b a5 1c 88 c2 8b 01 eb fe 48 83 7d 50 00 74 0c 0f
RIP [] :jbd2:jbd2_journal_commit_transaction+0x50/0x1074
RSP
<0>Kernel panic - not syncing: Fatal exception
2.2. BUG 描述
Bug 735768 hitting J_ASSERT(journal->j_running_transaction != NULL) in journal_commit_transaction (el)
2.3. rhel/ol 5 修復版本
2.4. rhel/ol 6 修復版本
2.6.32-304.el6 或 el/ol 6.4(2.6.32-358.el6) (ol, el)
[fs] jbd2: fix fsync() tid wraparound bug (Dave Wysochanski) [735768]
[fs] jbd: fix fsync() tid wraparound bug (Dave Wysochanski) [735768]
[fs] jbd, jbd2: fixed typos (Dave Wysochanski) [735768]
2.5. ol uek 修復版本
2.6.39 (1, 2, 3 沒找到)
2.6. ol uek 3 修復版本
3.8.13-16 (1, 2, 3)
3. 解決方案
Oracle Linux 6.x 系統(tǒng)升級到 6.4+，安裝 uek3 內核 。ol 5.x 建議重裝為 6.5 操作系統(tǒng)，安裝 uek3 內核。
RHEL 系統(tǒng)升級到 6.5+ 或 5.11+。
4. 系統(tǒng)升級方法
4.1. ol 6.x 升級到 6.5
# 配置 ol 6.5 的 yum repos
vi /etc/yum.repos.d/.....
yum clean all
yum update -x "Percona-Server-*"
# 注意升級完后時區(qū)變成系統(tǒng)默認的 EST，請手工修改為 CST
URL=ftp://aa.bb.cc.dd/.../linux/uek3
rpm -q xenstoreprovider && rpm -Uvh "$URL/xenstoreprovider-3.0-11.el6.x86_64.rpm"
rpm -ev kmod-ovmapi-uek
KV=3.8.13-55.1.5.el6uek
rpm -Uvh \
"$URL/kernel-uek-$KV.x86_64.rpm" \
"$URL/kernel-uek-devel-$KV.x86_64.rpm" \
"$URL/kernel-uek-firmware-$KV.noarch.rpm" \
"$URL/libdtrace-ctf-0.4.1-1.x86_64.rpm"
reboot
4.2. ol 5.x 升級到 6.5
備份數據
重裝系統(tǒng)
升級 el5 老版本軟件
4.3. ol 6.4 升級 uek3 內核
# 配置 6.5 yum
vi /etc/yum.repos.d/.....
yum -y update dracut
yum -y install elfutils-libs
URL=ftp://aa.bb.cc.dd/.../linux/uek3
rpm -q xenstoreprovider && rpm -Uvh "$URL/xenstoreprovider-3.0-11.el6.x86_64.rpm"
KV=3.8.13-55.1.5.el6uek
rpm -Uvh \
"$URL/kernel-uek-$KV.x86_64.rpm" \
"$URL/kernel-uek-devel-$KV.x86_64.rpm" \
"$URL/kernel-uek-firmware-$KV.noarch.rpm" \
"$URL/libdtrace-ctf-0.4.1-1.x86_64.rpm"
reboot
總結
以上是生活随笔為你收集整理的jbd2 mysql_jbd2 BUG · xiewen/xiewen.github.io Wiki · GitHub的全部內容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： .toint
- 下一篇： linux cmake编译源码,linu