C program to print string tokens

In C, string tokenization is the process of breaking a string into smaller parts (tokens) based on specified delimiters. The strtok() function from the string.h library is commonly used for this purpose.

Syntax

char *strtok(char *str, const char *delim);

Parameters

• str − The string to be tokenized (NULL for subsequent calls)
• delim − A string containing the delimiter characters

How It Works

The strtok() function works by following these steps −

• First call: Pass the string and delimiter to get the first token
• Subsequent calls: Pass NULL as the first argument to continue tokenizing the same string
• Returns NULL when no more tokens are found

Example: Basic String Tokenization

This example demonstrates how to tokenize a sentence using space as delimiter −

#include <stdio.h>
#include <string.h>

int main() {
    char s[] = "Let us see some string tokenizing fun";
    char* token = strtok(s, " ");
    
    while (token != NULL) {
        printf("%s<br>", token);
        token = strtok(NULL, " ");
    }
    return 0;
}

Let
us
see
some
string
tokenizing
fun

Example: Multiple Delimiters

You can use multiple delimiters by including them in the delimiter string −

#include <stdio.h>
#include <string.h>

int main() {
    char s[] = "apple,banana;orange:grape";
    char* token = strtok(s, ",;:");
    
    while (token != NULL) {
        printf("%s<br>", token);
        token = strtok(NULL, ",;:");
    }
    return 0;
}

apple
banana
orange
grape

Key Points

• The strtok() function modifies the original string by replacing delimiters with null terminators
• Use a copy of the string if you need to preserve the original
• The function is not thread-safe; use strtok_r() for multi-threaded applications

Conclusion

The strtok() function provides a simple and effective way to tokenize strings in C. It's particularly useful for parsing CSV data, command-line arguments, or any delimited text format.