Text Conversions

Boost.Locale provides several functions for basic string manipulation:

- to_upper: convert a string to upper case
- to_lower: convert a string to lower case
- to_title: convert a string to title case
- fold_case: makes a string case-agnostic (see case folding)
- normalize: convert equivalent code points to a consistent binary form (normalization)

All these functions receive an `std::locale` object as parameter or use a global locale by default.

Global locale is used in all examples below.

# Case Handing

For example:

std::string grussen = "grüßEN";
std::cout <<"Upper "<< boost::locale::to_upper(grussen) << std::endl
<<"Lower "<< boost::locale::to_lower(grussen) << std::endl
<<"Title "<< boost::locale::to_title(grussen) << std::endl
<<"Fold "<< boost::locale::fold_case(grussen) << std::endl;
boost::locale::to_title
std::basic_string< CharType > to_title(const CharType *begin, const CharType *end, const std::locale &loc=std::locale())
Definition: conversion.hpp:180
boost::locale::to_upper
std::basic_string< CharType > to_upper(const CharType *begin, const CharType *end, const std::locale &loc=std::locale())
Definition: conversion.hpp:120
boost::locale::to_lower
std::basic_string< CharType > to_lower(const CharType *begin, const CharType *end, const std::locale &loc=std::locale())
Definition: conversion.hpp:150
boost::locale::fold_case
std::basic_string< CharType > fold_case(const CharType *begin, const CharType *end, const std::locale &loc=std::locale())
Definition: conversion.hpp:210

Would print:

    Upper GRÜSSEN
    Lower grüßen
    Title Grüßen
    Fold  grüssen

There are existing functions `to_upper` and `to_lower` in the Boost.StringAlgo library, however the Boost.Locale functions operate on an entire string instead of performing incorrect character-by-character conversions.

For example:

std::wstring grussen = L"grüßen";
std::wcout << boost::algorithm::to_upper_copy(grussen) << " " << boost::locale::to_upper(grussen) << std::endl;

Would give in output:

    GRÜßEN GRÜSSEN

Where a letter "ß" was not converted correctly to double-S in first case because of a limitation of `std::ctype` facet.

This is even more problematic in case of UTF-8 encodings where non US-ASCII are not converted at all. For example, this code:

std::string grussen = "grüßen";
std::cout << boost::algorithm::to_upper_copy(grussen) << " " << boost::locale::to_upper(grussen) << std::endl;

Would modify ASCII characters only

    GRüßEN GRÜSSEN

# Unicode Normalization

Unicode normalization is the process of converting strings to a standard form, suitable for text processing and comparison. For example, character "ü" can be represented by a single code point or a combination of the character "u" and the diaeresis "¨". Normalization is an important part of Unicode text processing.

Unicode defines four normalization forms. Each specific form is selected by a flag passed to normalize function:

- NFD - Canonical decomposition - boost::locale::norm_nfd
- NFC - Canonical decomposition followed by canonical composition - boost::locale::norm_nfc or boost::locale::norm_default
- NFKD - Compatibility decomposition - boost::locale::norm_nfkd
- NFKC - Compatibility decomposition followed by canonical composition - boost::locale::norm_nfkc

For more details on normalization forms, read this report on unicode.org.

# Notes

- normalize operates only on Unicode-encoded strings, i.e.: UTF-8, UTF-16 and UTF-32 depending on the character width. So be careful when using non-UTF encodings as they may be treated incorrectly.
- fold_case is generally a locale-independent operation, but it receives a locale as a parameter to determine the 8-bit encoding.
- All of these functions can work with an STL string, a NULL terminated string, or a range defined by two pointers. They always return a newly created STL string.
- The length of the string may change; see the above example.

- © Copyright 2009-2012 Artyom Beilis, Distributed under the Boost Software License, Version 1.0.