下面的說明摘自于最新的Linux內核2.6.29，說明了strtok()這個函數已經不再使用，由速度更快的strsep()代替

/*

* linux/lib/string.c

*

* Copyright (C) 1991, 1992 Linus Torvalds

*/

/*

* stupid library routines.. The optimized versions should generally be found

* as inline code in

*

* These are buggy as well..

*

* * Fri Jun 25 1999, Ingo Oeser

* - Added strsep() which will replace strtok() soon (because strsep() is

* reentrant and should be faster). Use only strsep() in new code, please.

*

* * Sat Feb 09 2002, Jason Thomas ,

* Matthew Hawkins

* - Kissed strtok() goodbye

*/

strtok()這個函數大家都應該碰到過,但好像總有些問題, 這里著重講下它

下面我們來看一個例子：

int main() {

char test1[] = "feng,ke,wei";

char *test2 = "feng,ke,wei";

char *p; p = strtok(test1, ",");

while(p)

{

printf("%s\n", p);

p = strtok(NULL, ",");

}

return 0;

}

運行結果:

feng

ke

wei

但如果用p = strtok(test2, ",")則會出現內存錯誤,這是為什么呢?是不是跟它里面那個靜態變量有關呢? 我們來看看它的原碼:

/***

*strtok.c - tokenize a string with given delimiters

*

*?????? Copyright (c) Microsoft Corporation. All rights reserved.

*

*Purpose:

*?????? defines strtok() - breaks string into series of token

*?????? via repeated calls.

*

*******************************************************************************/

#include

#include

#ifdef _MT

#include

#endif /* _MT */

/***

*char *strtok(string, control) - tokenize string with delimiter in control

*

*Purpose:

*?????? strtok considers the string to consist of a sequence of zero or more

*?????? text tokens separated by spans of one or more control chars. the first

*?????? call, with string specified, returns a pointer to the first char of the

*?????? first token, and will write a null char into string immediately

*?????? following the returned token. subsequent calls with zero for the first

*?????? argument (string) will work thru the string until no tokens remain. the

*?????? control string may be different from call to call. when no tokens remain

*?????? in string a NULL pointer is returned. remember the control chars with a

*?????? bit map, one bit per ascii char. the null char is always a control char.

*??????//這里已經說得很詳細了!!比MSDN都好! *Entry:

*?????? char *string - string to tokenize, or NULL to get next token

*?????? char *control - string of characters to use as delimiters

*

*Exit:

*?????? returns pointer to first token in string, or if string

*?????? was NULL, to next token

*?????? returns NULL when no more tokens remain.

*

*Uses:

*

*Exceptions:

*

*******************************************************************************/

char * __cdecl strtok (

char * string,

const char * control

)

{

unsigned char *str;

const unsigned char *ctrl = control;

unsigned char map[32];

int count;

#ifdef _MT

_ptiddata ptd = _getptd();

#else /* _MT */

static char *nextoken;?????????????????????? //保存剩余子串的靜態變量

#endif /* _MT */

/* Clear control map */

for (count = 0; count < 32; count++)

map[count] = 0;

/* Set bits in delimiter table */

do {

map[*ctrl >> 3] |= (1 << (*ctrl & 7));

} while (*ctrl++);

/* Initialize str. If string is NULL, set str to the saved

* pointer (i.e., continue breaking tokens out of the string

* from the last strtok call) */

if (string)

str = string;???????????????????????????? //第一次調用函數所用到的原串

else

#ifdef _MT

str = ptd->_token;

#else /* _MT */

str = nextoken;????????????????????? //將函數第一參數設置為NULL時調用的余串

#endif /* _MT */

/* Find beginning of token (skip over leading delimiters). Note that

* there is no token iff this loop sets str to point to the terminal

* null (*str == '\0') */

while ( (map[*str >> 3] & (1 << (*str & 7))) && *str )

str++;

string = str;????????????????????????????????? //此時的string返回余串的執行結果

/* Find the end of the token. If it is not the end of the string,

* put a null there. */

//這里就是處理的核心了, 找到分隔符,并將其設置為'\0',當然'\0'也將保存在返回的串中

for ( ; *str ; str++ )

if ( map[*str >> 3] & (1 << (*str & 7)) ) {

*str++ = '\0';????????????? //這里就相當于修改了串的內容① ??????????????????????? break;

}

/* Update nextoken (or the corresponding field in the per-thread data

* structure */

#ifdef _MT

ptd->_token = str;

#else /* _MT */

nextoken = str;???????????????? //將余串保存在靜態變量中,以便下次調用 #endif /* _MT */

/* Determine if a token has been found. */

if ( string == str )

return NULL;

else

return string;1. strtok介紹眾所周知，strtok可以根據用戶所提供的分割符(同時分隔符也可以為復數比如“，。”)

將一段字符串分割直到遇到"\0".

比如，分隔符=“，” 字符串=“Fred，John，Ann”

通過strtok 就可以把3個字符串 “Fred”???? “John”????? “Ann”提取出來。

上面的C代碼為

QUOTE:

int in=0;

char buffer[]="Fred,John,Ann"

char *p[3];

char *buff = buffer;

while((p[in]=strtok(buf,","))!=NULL) {

i++;

buf=NULL; }

如上代碼，第一次執行strtok需要以目標字符串的地址為第一參數(buf=buffer)，之后strtok需要以NULL為第一參數 (buf=NULL)。指針列p[],則儲存了分割后的結果，p[0]="John",p[1]="John",p[2]="Ann"，而buf就變 成??? Fred\0John\0Ann\0。

2. strtok的弱點

讓我們更改一下我們的計劃：我們有一段字符串 "Fred male 25,John male 62,Anna female 16" 我們希望把這個字符串整理輸入到一個struct，

QUOTE:

struct person {

char [25] name ;

char [6] sex;

char [4] age;

}

要做到這個，其中一個方法就是先提取一段被“，”分割的字符串，然后再將其以“ ”(空格)分割。

比如： 截取 "Fred male 25" 然后分割成 "Fred" "male" "25"

以下我寫了個小程序去表現這個過程：

QUOTE:

#include

#include

#define INFO_MAX_SZ 255

int main()

{

int in=0;

char buffer[INFO_MAX_SZ]="Fred male 25,John male 62,Anna female 16";

char *p[20];

char *buf=buffer;

while((p[in]=strtok(buf,","))!=NULL) {

buf=p[in];

while((p[in]=strtok(buf," "))!=NULL) {

in++;

buf=NULL;

}

p[in++]="***"; //表現分割

buf=NULL; }

printf("Here we have %d strings\n",i);

for (int j=0; j

printf(">%s

return 0;

}

這個程序輸出為：

Here we have 4 strings

>Fred<

>male<

>25<

>***<

這只是一小段的數據，并不是我們需要的。但這是為什么呢？ 這是因為strtok使用一個static(靜態)指針來操作數據，讓我來分析一下以上代碼的運行過程：

紅色為strtok的內置指針指向的位置，藍色為strtok對字符串的修改

1. "Fred male 25,John male 62,Anna female 16" //外循環

2. "Fred male 25\0John male 62,Anna female 16" //進入內循環

3.??? "Fred\0male 25\0John male 62,Anna female 16"

4.??? "Fred\0male\025\0John male 62,Anna female 16"

5 "Fred\0male\025\0John male 62,Anna female 16" //內循環遇到"\0"回到外循環

6?? "Fred\0male\025\0John male 62,Anna female 16" //外循環遇到"\0"運行結束。

3. 使用strtok_r

在這種情況我們應該使用strtok_r, strtok reentrant.

char *strtok_r(char *s, const char *delim, char **ptrptr);

相對strtok我們需要為strtok提供一個指針來操作，而不是像strtok使用配套的指針。

代碼：

QUOTE:

#include

#include

#define INFO_MAX_SZ 255

int main()

{

int in=0;

char buffer[INFO_MAX_SZ]="Fred male 25,John male 62,Anna female 16";

char *p[20];

char *buf=buffer;

char *outer_ptr=NULL;

char *inner_ptr=NULL;

while((p[in]=strtok_r(buf,",",&outer_ptr))!=NULL) {

buf=p[in];

while((p[in]=strtok_r(buf," ",&inner_ptr))!=NULL) {

in++;

buf=NULL;

}

p[in++]="***";

buf=NULL; }

printf("Here we have %d strings\n",i);

for (int j=0; jn

printf(">%s

return 0;

}

這一次的輸出為：

Here we have 12 strings

>Fred<

>male<

>25<

>***<

>John<

>male<

>62<

>***<

>Anna<

>female<

>16<

>***<

讓我來分析一下以上代碼的運行過程：

紅色為strtok_r的outer_ptr指向的位置，

紫色為strtok_r的inner_ptr指向的位置，

藍色為strtok對字符串的修改

1. "Fred male 25,John male 62,Anna female 16" //外循環

2. "Fred male 25\0John male 62,Anna female 16"//進入內循環

3.?? "Fred\0male 25\0John male 62,Anna female 16"

4?? "Fred\0male\025\0John male 62,Anna female 16"

5 "Fred\0male\025\0John male 62,Anna female 16" //內循環遇到"\0"回到外循環

6?? "Fred\0male\025\0John male 62\0Anna female 16"//進入內循環

}

原來, 該函數修改了原串.

所以,當使用char *test2 = "feng,ke,wei"作為第一個參數傳入時,在位置①處, 由于test2指向的內容保存在文字常量區,該區的內容是不能修改的,所以會出現內存錯誤. 而char test1[] = "feng,ke,wei" 中的test1指向的內容是保存在棧區的,所以可以修改.

看到這里 大家應該會對文字常量區有個更加理性的認識吧.....

總結

以上是生活随笔為你收集整理的mysql strtok,strtok()和strtok_r()的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。