strtok - PHP code example

PHP Function Sttok - How to Tokenize a String

The php function strtok is used to tokenize a string into smaller parts on the basis of given delimiters. It takes input String as a first parameter and delimiters as a second parameter. It keeps track of its position in the string between calls, so it can return the next text token to be found. It also works well with regular expressions, which can be useful when working with data sets.

The main difference between this and other string splitting functions is that it returns just one part of the split string at a time rather than all of it in an array, so it's potentially more economic and memory-preserving for large strings. In addition, it can be used with regular expressions, which provide more flexibility and control when splitting strings.

Tokenizing a string means breaking it up into individual pieces or "tokens." The example below shows how to split the phrase "Hey Buddy! Welcome to PavanKumarSake Park" into the words "Hey Buddy!" and "Welcome to." Using regular expressions, you can expand on this by adding patterns that will identify the individual words. This is a very powerful technique for working with structured data.

strtok example

throw new \InvalidArgumentException(sprintf('Source path %s has to start with a slash.', $path));
    }
    // If no language is explicitly specified we default to the current URL     // language. If we used a language different from the one conveyed by the     // requested URL, we might end up being unable to check if there is a path     // alias matching the URL path.     $langcode = $langcode ?: $this->languageManager->getCurrentLanguage(LanguageInterface::TYPE_URL)->getId();

    // Check the path whitelist, if the top-level part before the first /     // is not in the list, then there is no need to do anything further,     // it is not in the database.     if ($path === '/' || !$this->whitelist->get(strtok(trim($path, '/'), '/'))) {
      return $path;
    }

    // During the first call to this method per language, load the expected     // paths for the page from cache.     if (empty($this->langcodePreloaded[$langcode])) {
      $this->langcodePreloaded[$langcode] = TRUE;
      $this->lookupMap[$langcode] = [];

      // Load the cached paths that should be used for preloading. This only       // happens if a cache key has been set.

    asort($ua_langcodes);
    foreach ($ua_langcodes as $langcode => $qvalue) {
      // For Chinese languages the generic tag is either zh-hans or zh-hant, so       // we need to handle this separately, we can not split $langcode on the       // first occurrence of '-' otherwise we get a non-existing language zh.       // All other languages use a langcode without a '-', so we can safely       // split on the first occurrence of it.       if (strlen($langcode) > 7 && (substr($langcode, 0, 7) == 'zh-hant' || substr($langcode, 0, 7) == 'zh-hans')) {
        $generic_tag = substr($langcode, 0, 7);
      }
      else {
        $generic_tag = strtok($langcode, '-');
      }
      if (!empty($generic_tag) && !isset($ua_langcodes[$generic_tag])) {
        // Add the generic langcode, but make sure it has a lower qvalue as the         // more specific one, so the more specific one gets selected if it's         // defined by both the user agent and us.         $ua_langcodes[$generic_tag] = $qvalue - 0.1;
      }
    }

    // Find the added language with the greatest qvalue, following the rules     // of RFC 2616 (section 14.4). If several languages have the same qvalue,

        if (empty(self::$_ldml[(string) $locale])) {
            $filename = dirname(__FILE__) . '/Data/' . $locale . '.xml';
            if (!file_exists($filename)) {
                throw new Zend_Locale_Exception("Missing locale file '$filename' for '$locale' locale.");
            }

            self::$_ldml[(string) $locale] = simplexml_load_string(file_get_contents($filename));
        }

        // search for 'alias' tag in the search path for redirection         $search = '';
        $tok = strtok($path, '/');

        // parse the complete path         if (!empty(self::$_ldml[(string) $locale])) {
            while ($tok !== false) {
                $search .= '/' . $tok;
                if (strpos($search, '[@') !== false) {
                    while (strrpos($search, '[@') > strrpos($search, ']')) {
                        $tok = strtok('/');
                        if (empty($tok)) {
                            $search .= '/';
                        }

$active_ancestor_item_ids[] = $_anc_id;
            }

            $active_parent_item_ids[] = (int) $menu_item->menu_item_parent;

            // If the menu item corresponds to the currently requested URL.         } elseif ( 'custom' === $menu_item->object && isset( $_SERVER['HTTP_HOST'] ) ) {
            $_root_relative_current = untrailingslashit( $_SERVER['REQUEST_URI'] );

            // If it's the customize page then it will strip the query var off the URL before entering the comparison block.             if ( is_customize_preview() ) {
                $_root_relative_current = strtok( untrailingslashit( $_SERVER['REQUEST_URI'] ), '?' );
            }

            $current_url        = set_url_scheme( 'http://' . $_SERVER['HTTP_HOST'] . $_root_relative_current );
            $raw_item_url       = strpos( $menu_item->url, '#' ) ? substr( $menu_item->url, 0, strpos( $menu_item->url, '#' ) ) : $menu_item->url;
            $item_url           = set_url_scheme( untrailingslashit( $raw_item_url ) );
            $_indexless_current = untrailingslashit( preg_replace( '/' . preg_quote( $wp_rewrite->index, '/' ) . '$/', '', $current_url ) );

            $matches = array(
                $current_url,
                urldecode( $current_url ),
                $_indexless_current,

$this->executableFinder = new ExecutableFinder();
    }

    /** * Finds The PHP executable. */
    public function find(bool $includeArgs = true): string|false
    {
        if ($php = getenv('PHP_BINARY')) {
            if (!is_executable($php)) {
                $command = '\\' === \DIRECTORY_SEPARATOR ? 'where' : 'command -v';
                if (\function_exists('exec') && $php = strtok(exec($command.' '.escapeshellarg($php)), \PHP_EOL)) {
                    if (!is_executable($php)) {
                        return false;
                    }
                } else {
                    return false;
                }
            }

            if (@is_dir($php)) {
                return false;
            }

$scripts->add( 'mediaelement', false, array( 'jquery', 'mediaelement-core', 'mediaelement-migrate' ), '4.2.17', 1 );
    $scripts->add( 'mediaelement-core', "/wp-includes/js/mediaelement/mediaelement-and-player$suffix.js", array(), '4.2.17', 1 );
    $scripts->add( 'mediaelement-migrate', "/wp-includes/js/mediaelement/mediaelement-migrate$suffix.js", array(), false, 1 );

    did_action( 'init' ) && $scripts->add_inline_script(
        'mediaelement-core',
        sprintf(
            'var mejsL10n = %s;',
            wp_json_encode(
                array(
                    'language' => strtolower( strtok( determine_locale(), '_-' ) ),
                    'strings'  => array(
                        'mejs.download-file'       => __( 'Download File' ),
                        'mejs.install-flash'       => __( 'You are using a browser that does not have Flash player enabled or installed. Please turn on your Flash player plugin or download the latest version from https://get.adobe.com/flashplayer/' ),
                        'mejs.fullscreen'          => __( 'Fullscreen' ),
                        'mejs.play'                => __( 'Play' ),
                        'mejs.pause'               => __( 'Pause' ),
                        'mejs.time-slider'         => __( 'Time Slider' ),
                        'mejs.time-help-text'      => __( 'Use Left/Right Arrow keys to advance one second, Up/Down arrows to advance ten seconds.' ),
                        'mejs.live-broadcast'      => __( 'Live Broadcast' ),
                        'mejs.volume-help-text'    => __( 'Use Up/Down Arrow keys to increase or decrease volume.' ),
                        'mejs.unmute'              => __( 'Unmute' ),

if (@is_file($file = $dir.\DIRECTORY_SEPARATOR.$name.$suffix) && ('\\' === \DIRECTORY_SEPARATOR || @is_executable($file))) {
                    return $file;
                }

                if (!@is_dir($dir) && basename($dir) === $name.$suffix && @is_executable($dir)) {
                    return $dir;
                }
            }
        }

        $command = '\\' === \DIRECTORY_SEPARATOR ? 'where' : 'command -v';
        if (\function_exists('exec') && ($executablePath = strtok(@exec($command.' '.escapeshellarg($name)), \PHP_EOL)) && is_executable($executablePath)) {
            return $executablePath;
        }

        return $default;
    }
}

$tablefield_type_without_parentheses = preg_replace(
                '/'
                . '(.+)'       // Field type, e.g. `int`.                 . '$\d*$'    // Display width.                 . '(.*)'       // Optional attributes, e.g. `unsigned`.                 . '/',
                '$1$2',
                $tablefield_type_lowercased
            );

            // Get the type without attributes, e.g. `int`.             $tablefield_type_base = strtok( $tablefield_type_without_parentheses, ' ' );

            // If the table field exists in the field array...             if ( array_key_exists( $tablefield_field_lowercased, $cfields ) ) {

                // Get the field type from the query.                 preg_match( '|`?' . $tablefield->Field . '`? ([^ ]*( unsigned)?)|i', $cfields[ $tablefield_field_lowercased ], $matches );
                $fieldtype            = $matches[1];
                $fieldtype_lowercased = strtolower( $fieldtype );

                $fieldtype_without_parentheses = preg_replace(
                    '/'
                    .

        return str_starts_with($path, '/store-api');
    }

    private function isAdminApi(string $path): bool
    {
        return str_starts_with($path, '/api');
    }

    private function isRepositoryCrudRoute(Route $route): bool
    {
        $controllerClass = strtok($route->getDefault('_controller'), ':');

        return $controllerClass === ApiController::class;
    }

    private function isCoreRoute(Route $route): bool
    {
        $controllerClass = (string) strtok((string) $route->getDefault('_controller'), ':');

        return str_starts_with($controllerClass, 'Shopware\Core');
    }

/** * Get default button labels. * * Provides an array of the default button labels based on the mime type of the current control. * * @since 4.9.0 * * @return string[] An associative array of default button labels keyed by the button name. */
    public function get_default_button_labels() {
        // Get just the mime type and strip the mime subtype if present.         $mime_type = ! empty( $this->mime_type ) ? strtok( ltrim( $this->mime_type, '/' ), '/' ) : 'default';

        switch ( $mime_type ) {
            case 'video':
                return array(
                    'select'       => __( 'Select video' ),
                    'change'       => __( 'Change video' ),
                    'default'      => __( 'Default' ),
                    'remove'       => __( 'Remove' ),
                    'placeholder'  => __( 'No video selected' ),
                    'frame_title'  => __( 'Select video' ),
                    'frame_button' => __( 'Choose video' ),
                );

$queries[] = new MatchQuery($searchField, $token, ['boost' => 5 * $ranking]);
                $queries[] = new MatchPhrasePrefixQuery($searchField, $token, ['boost' => $ranking, 'slop' => 5]);
                $queries[] = new WildcardQuery($searchField, '*' . $token . '*');

                if ($item['tokenize']) {
                    $queries[] = new MatchQuery($searchField, $token, ['fuzziness' => 'auto', 'boost' => 3 * $ranking]);
                    $queries[] = new MatchQuery($ngramField, $token);
                }

                if (str_contains($item['field'], '.') && !str_contains($item['field'], 'customFields')) {
                    $nested = strtok($item['field'], '.');

                    foreach ($queries as $query) {
                        $tokenBool->add(new NestedQuery($nested, $query), BoolQuery::SHOULD);
                    }

                    continue;
                }

                foreach ($queries as $query) {
                    $tokenBool->add($query, BoolQuery::SHOULD);
                }
            }

$this->_socket = @fsockopen($host, $port, $errno, $errstr, self::TIMEOUT_CONNECTION);
        if (!$this->_socket) {
            /** * @see Zend_Mail_Protocol_Exception */
            throw new Zend_Mail_Protocol_Exception('cannot connect to host; error = ' . $errstr .
                                                   ' (errno = ' . $errno . ' )');
        }

        $welcome = $this->readResponse();

        strtok($welcome, '<');
        $this->_timestamp = strtok('>');
        if (!strpos($this->_timestamp, '@')) {
            $this->_timestamp = null;
        } else {
            $this->_timestamp = '<' . $this->_timestamp . '>';
        }

        if ($ssl === 'TLS') {
            $this->request('STLS');
            $result = stream_socket_enable_crypto($this->_socket, true, $this->getCryptoMethod());
            if (!$result) {

$this->executableFinder = new ExecutableFinder();
    }

    /** * Finds The PHP executable. */
    public function find(bool $includeArgs = true): string|false
    {
        if ($php = getenv('PHP_BINARY')) {
            if (!is_executable($php)) {
                $command = '\\' === \DIRECTORY_SEPARATOR ? 'where' : 'command -v';
                if ($php = strtok(exec($command.' '.escapeshellarg($php)), \PHP_EOL)) {
                    if (!is_executable($php)) {
                        return false;
                    }
                } else {
                    return false;
                }
            }

            if (@is_dir($php)) {
                return false;
            }

    public static function splitMessage($message, &$headers, &$body, $EOL = Zend_Mime::LINEEND)
    {
        // check for valid header at first line         $firstline = strtok($message, "\n");
        if (!preg_match('%^[^\s]+[^:]*:%', $firstline)) {
            $headers = array();
            // TODO: we're ignoring \r for now - is this function fast enough and is it safe to asume noone needs \r?             $body = str_replace(array("\r", "\n"), array('', $EOL), $message);
            return;
        }

        // find an empty line between headers and body         // default is set new line         if (strpos($message, $EOL . $EOL)) {
            list($headers, $body) = explode($EOL . $EOL, $message, 2);

    $files = $archive->listContents();
    if (!$files) {
      $this->messenger()->addError($this->t('Provided archive contains no files.'));
      return;
    }

    // Unfortunately, we can only use the directory name to determine the     // project name. Some archivers list the first file as the directory (i.e.,     // MODULE/) and others list an actual file (i.e., MODULE/README.TXT).     $project = strtok($files[0], '/\\');

    $archive_errors = $this->moduleHandler->invokeAll('verify_update_archive', [$project, $local_cache, $directory]);
    if (!empty($archive_errors)) {
      $this->messenger()->addError(array_shift($archive_errors));
      // @todo: Fix me in D8: We need a way to set multiple errors on the same       // form element and have all of them appear!       if (!empty($archive_errors)) {
        foreach ($archive_errors as $error) {
          $this->messenger()->addError($error);
        }
      }