Programming assignment in C

[Tankbuster]

Hi everybody, I'm having a little problem with some C work I need to take care of. I'm just learning that language, and I want to build a so-called tokenizer, i.e. a program c that reads a char* of text, splits it up into several char* according to certain delimiters (space, comma, question mark, etcetera), and finally assigns a char** as a pointer-reference to the various individual parts.

For example, the input char* "This is just a test!", would be converted into a char** with arrays "This", "is", "just", "a", "test".

Basically this would be the header of the program: char** tokenize (char* input, int* arraylength)
I already have a function "int is_delimiter (char input)" which returns 1 if the char is a delimiter and 0 if it's not.

However I have no idea what to do now. I guess I need to make a for-loop going over every char of my input, but then how do I return a char** that points to the various parts...

Any help would be much appreciated

EDIT: Oh by the way, I'm well aware that there is an in-built function in C to do this for you, but that's not the purpose of the exercise.

[Juvenal]

...Basically this would be the header of the program: char** tokenize (char* input, int* arraylength)
I already have a function "int is_delimiter (char input)" which returns 1 if the char is a delimiter and 0 if it's not.

However I have no idea what to do now. I guess I need to make a for-loop going over every char of my input, but then how do I return a char** that points to the various parts...

Any help would be much appreciated

EDIT: Oh by the way, I'm well aware that there is an in-built function in C to do this for you, but that's not the purpose of the exercise.

Glad you noticed the existence of strtok.

You need to add the delimiter to be searched for as another parameter to your function. It would make things a lot simpler if the input string was a string (i.e. delimited with a trailing ASCII zero). This is because when you split it into token strings, you need space at the end to put the last zero that terminates the last token.

The simplest way to split a string into tokens is to replace the specified delimiter with ASCII zero, this automatically makes the tokens into C strings. You can build an array of char pointers to point to them, and return this array to the caller.

So, you have to step through the input string twice, firstly counting the number of delimiters so you can allocate your pointer array the right size, then again - substituting delimiters with zero and filling out your pointer array.

You can then return the address of the pointer array, and the caller can be responsible for freeing it when they're done.

[Tankbuster]

Hi Juvenal, thanks for helping me out
I'm don't need to give the delimiters as a specific argument because I'm using the same delimiters over and over again. As I said, I've already defined af function is_delimiter which applies to my whole program and returns 1 if the char in question is in fact a delimiter.

This is my current program:

Spoiler Alert, click show to read: 

#include<stdio.h>

#include<string.h>

#include"tokenize.h"

#include<stdlib.h>

main() {
char** output;
char input[];
int numberofwords=0;
int i;
int beginword;

for(i=0;scanf("%s",input)==1;++i); /*reading words*/

w[i]=calloc(strlen(input)+1,sizeof(char));
strcpy(w[i],input);
i=0;

for(i=0;i<strlen(input);++i) {

if(isdelimiter(input[i]) i++;

else {

numberwords++;
while(isdelimiter!=1) i++;
}

(First for-lus to determine number of words is done)

for(i=0;i<strlen(input);++i) {

if(isdelimiter(input[i]) {
i++;
}

else {

beginword=i;
while(isdelimiter!=1) i++;
/*Now there is a valid word in the space [beginword,i], but how do I assign it to output?*/

}

}

}

Basically the problem is that I don't know how to assign that specific series of chars to the char** that I have in place

[Garnier]

I've never used straight C before, in C++ we have the "new" keyword. So this might not be good C code, but it does work.

I use malloc. Here's a quick example of assigning a string literal to a malloc-ed char**. There may be an easier way to do this but I don't know.

char** GetStr()
{
    int numStrings = 4;
    char** str = (char**)malloc(numStrings);

    char* temp0 = "hello0";
    str[0] = (char*)malloc(strlen(temp0));
    strcpy(str[0],temp0);

    char* temp1 = "hello1";
    str[1] = (char*)malloc(strlen(temp1));
    strcpy(str[1],temp1);

    
    return str;
}

When this returns, str should look like this:

str[0] == "hello0"
str[1] == "hello1"
str[2] == NULL
str[3] == NULL

To copy your strings, you can use a for loop.

Edit: Or you can use Juvenal's method which I just read, which is nicer. :p

By the way, how'd you get the color on your code, did you do that manually?

[Tankbuster]

Hmmm, not sure if that works... I'm trying right now and getting lots of errors that I can't seem to sort out

The colour comes directly from Visual Studio C/C++ Express 2008, by the way. As it turns out TWC is quite happy to copy that...

[Garnier]

Ok, I use VS 6.0. Maybe I did something that is considered evil by 2008. :/

Edit:
I went ahead and tried it in VS C++ 2008 Express and it worked, so I don't know what's wrong. This is the contents of my program.cpp:

Spoiler Alert, click show to read: 

#include "stdafx.h"
#include<stdio.h>
#include<string.h>
#include<stdlib.h>

char** GetStr()
{
char** str = (char**)malloc(10);

char* temp0 = "hello0";
str[0] = (char*)malloc(strlen(temp0));
strcpy(str[0],temp0);

char* temp1 = "hello1";
str[1] = (char*)malloc(strlen(temp1));
strcpy(str[1],temp1);

return str;
}

int main()
{
char** test = GetStr();

return 0;
}