Database changes have finished applying - please report any issues you're (still) seeing to support@shoutwiki.com.

Module:Unicode data

From TechInfoDepot
Jump to navigationJump to search
Documentation icon Module documentation[view] [edit] [history] [purge]

Template:High riskTemplate:Cascade-protected template

Usage

This module provides functions that access information on Unicode code points. The information is retrieved from data modules generated from the Unicode Character Database, or derived by rules given in the Unicode Specification. It and its submodules were copied from English Wiktionary and then modified; see there for more information.

Parameters and functions

code point

The code point is to be entered as hexadecimal value. For example, Expression error: Unrecognized punctuation character "[".:

|A9hex
|0xA9hex
|0x00A9hex
|0x00a9hex
{{#invoke:Unicode data|lookup|name|0x00A9}} → <reserved-00A9>

Incorrect or unintended results:

169dec: {{#invoke:Unicode data|lookup|name|169}} → <reserved-0169> Template:Nay —"U+00A9" &copy; expected; but is read as 00A9hex (that is, Template:Hex2decdec
U+00A9 {{#invoke:Unicode data|lookup|name|U+00A9}} Template:Nay —do not use "U+" prefix
غ {{#invoke:Unicode data|lookup|name|غ}} Template:Nay —cannot enter a character as codepoint

"lookup" and "is" functions

lookup, is
Template-invokable functions that allow access to the functions starting with lookup and is.For most of the functions, add the code point in hexadecimal base as the next parameter. For is|Latin, is|rtl, and is|valid_pagename, add character string. HTML character references in the text are decoded by the module into code points.
For example, {{#invoke:Unicode data|is|Latin|àzàhàr̃iyyā̀}}Lua error at line 297: attempt to index local 'data_module' (a boolean value)..
Internally, in modules, these functions are named using underscore: lookup_name|code pointlookup_name
For &A9; ©: {{#invoke:Unicode data|lookup|name|A9}} → <reserved-00A9>

Functions overview

Topic Function Parameter type
(string=by character(s); c.p. by 0xHex value)
Example Returns Character
Unicode character name |lookup|name code point Template:Ubl Template:Ubl Template:Ubl
Scripts |lookup|script code point lookup|script|A061}} Lua error at line 297: attempt to index local 'data_module' (a boolean value). Template:Ubl
Blocks |lookup|block code point lookup|block|A061}} Lua error at line 15: attempt to index local 'ranges' (a boolean value). Template:Ubl
Planes |lookup|plane code point Template:Ubl Template:Ubl Template:Ubl


General Category |lookup|category code point Template:Ubl Template:Ubl Template:Ubl
Controls |is|control code point Template:Ubl Template:Ubl Template:Ubl
Unihan properties |lookup|kCantonese code point Template:Ubl Template:Ubl Template:Ubl
Latin script |is|Latin string Template:Ubl Template:Ubl
WP:Article title (WP:NCTR) |is|valid_pagename string Template:Ubli Template:Ubl
Bidirectionality, right-to-left scripts |is|rtl string Template:Ubl Template:Ubli Template:Ubl
Combining character |is|combining code point Template:Ubl Template:Ubl Template:Ubl
Character assignation |is|assigned code point Template:Ubl Template:Ubl Template:Ubl
Printable |is|printable code point Template:Ubl Template:Ubl Template:Ubl
Template:Slink |is|whitespace code point Template:Ubl Template:Ubl Template:Ubl
Alias names |aliases [application unknown] Template:Ubl
Combining class | [application unknown] Template:Ubl
Age | [application unknown] Template:Ubl
get_best_script |get_best_script [application unknown] Template:Ubl

Data modules

The data used by functions in this module is found in submodules. Some are generated by AWK scripts shown at User:Kephir/Unicode on English Wiktionary, others by Lua scripts on the /make subpages of the submodules.

The name data modules (Module:Unicode data/names/xxx) were compiled from UnicodeData.txt. Each one contains, at maximum, code points U+xxx000 to U+xxxFFF. Script error: No such module "Unicode data/documentation functions".

Copyright

The Unicode database is released by Unicode Inc. under the following terms:

Copyright © 1991-2018 Unicode, Inc. All rights reserved. Distributed under the Terms of Use in https://www.unicode.org/copyright.html.

Permission is hereby granted, free of charge, to any person obtaining a copy of the Unicode data files and any associated documentation (the "Data Files") or Unicode software and any associated documentation (the "Software") to deal in the Data Files or Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, and/or sell copies of the Data Files or Software, and to permit persons to whom the Data Files or Software are furnished to do so, provided that either (a) this copyright and permission notice appear with all copies of the Data Files or Software, or (b) this copyright and permission notice appear in associated Documentation.

THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE DATA FILES OR SOFTWARE.

Except as contained in this notice, the name of a copyright holder shall not be used in advertising or otherwise to promote the sale, use or other dealings in these Data Files or Software without prior written authorization of the copyright holder.

Known issues

  • Reading data like Module:Unicode data/aliases not provided nor documented
  • Test fail: lookup_category U+FFFF (<noncharacter-FFFF>) expected: Cn.
{{#invoke:Unicode data|lookup|category|0xFFFF}}Lua error at line 297: attempt to index local 'data_module' (a boolean value). [Nil]

See also

  • Named entities: for example, Expression error: Unrecognized punctuation character "[".: {{#invoke:LoadData|Numcr2namecr|0x22C1}} → &bigvee;, &Vee;, &xvee;

  1 local p = {}
  2 
  3 local floor = math.floor
  4 
  5 local function errorf(level, ...)
  6 	if type(level) == "number" then
  7 		return error(string.format(...), level + 1)
  8 	else -- level is actually the format string.
  9 		return error(string.format(level, ...), 2)
 10 	end
 11 end
 12 
 13 local function binary_range_search(codepoint, ranges)
 14 	local low, mid, high
 15 	low, high = 1, ranges.length or require "Module:TableTools".length(ranges)
 16 	while low <= high do
 17 		mid = floor((low + high) / 2)
 18 		local range = ranges[mid]
 19 		if codepoint < range[1] then
 20 			high = mid - 1
 21 		elseif codepoint <= range[2] then
 22 			return range, mid
 23 		else
 24 			low = mid + 1
 25 		end
 26 	end
 27 	return nil, mid
 28 end
 29 p.binary_range_search = binary_range_search
 30 
 31 --[[
 32 local function linear_range_search(codepoint, ranges)
 33 	for i, range in ipairs(ranges) do
 34 		if range[1] <= codepoint and codepoint <= range[2] then
 35 			return range
 36 		end
 37 	end
 38 end
 39 --]]
 40 
 41 -- Load a module by indexing "loader" with the name of the module minus the
 42 -- "Module:Unicode data/" part. For instance, loader.blocks returns
 43 -- [[Module:Unicode data/blocks]]. If a module cannot be loaded, false will be
 44 -- returned.
 45 local loader = setmetatable({}, {
 46 	__index = function (self, key)
 47 		local success, data = pcall(mw.loadData, "Module:Unicode data/" .. key)
 48 		if not success then
 49 			data = false
 50 		end
 51 		self[key] = data
 52 		return data
 53 	end
 54 })
 55 
 56 -- For the algorithm used to generate Hangul Syllable names,
 57 -- see "Hangul Syllable Name Generation" in section 3.12 of the
 58 -- Unicode Specification:
 59 -- https://www.unicode.org/versions/Unicode11.0.0/ch03.pdf
 60 local name_hooks = {
 61 	{     0x00,     0x1F, "<control-%04X>" }, -- C0 control characters
 62 	{     0x7F,     0x9F, "<control-%04X>" }, -- DEL and C1 control characters
 63 	{   0x3400,   0x4DBF, "CJK UNIFIED IDEOGRAPH-%04X" }, -- CJK Ideograph Extension A
 64 	{   0x4E00,   0x9FFF, "CJK UNIFIED IDEOGRAPH-%04X" }, -- CJK Ideograph
 65 	{   0xAC00,   0xD7A3, function (codepoint) -- Hangul Syllables
 66 		local Hangul_data = loader.Hangul
 67 		local syllable_index = codepoint - 0xAC00
 68 
 69 		return ("HANGUL SYLLABLE %s%s%s"):format(
 70 			Hangul_data.leads[floor(syllable_index / Hangul_data.final_count)],
 71 			Hangul_data.vowels[floor((syllable_index % Hangul_data.final_count)
 72 				/ Hangul_data.trail_count)],
 73 			Hangul_data.trails[syllable_index % Hangul_data.trail_count]
 74 		)
 75 	end },
 76 	-- High Surrogates, High Private Use Surrogates, Low Surrogates
 77 	{   0xD800,   0xDFFF, "<surrogate-%04X>" },
 78 	{   0xE000,   0xF8FF, "<private-use-%04X>" }, -- Private Use
 79 	-- CJK Compatibility Ideographs
 80 	{   0xF900,   0xFA6D, "CJK COMPATIBILITY IDEOGRAPH-%04X" },
 81 	{   0xFA70,   0xFAD9, "CJK COMPATIBILITY IDEOGRAPH-%04X" },
 82 	{  0x17000,  0x187F7, "TANGUT IDEOGRAPH-%04X" }, -- Tangut Ideograph
 83 	{  0x18800,  0x18AFF, function (codepoint)
 84 		return ("TANGUT COMPONENT-%03d"):format(codepoint - 0x187FF)
 85 	end },
 86 	{  0x18D00,  0x18D08, "TANGUT IDEOGRAPH-%04X" }, -- Tangut Ideograph Supplement
 87 	{  0x1B170,  0x1B2FB, "NUSHU CHARACTER-%04X" }, -- Nushu
 88 	{  0x20000,  0x2A6DF, "CJK UNIFIED IDEOGRAPH-%04X" }, -- CJK Ideograph Extension B
 89 	{  0x2A700,  0x2B739, "CJK UNIFIED IDEOGRAPH-%04X" }, -- CJK Ideograph Extension C
 90 	{  0x2B740,  0x2B81D, "CJK UNIFIED IDEOGRAPH-%04X" }, -- CJK Ideograph Extension D
 91 	{  0x2B820,  0x2CEA1, "CJK UNIFIED IDEOGRAPH-%04X" }, -- CJK Ideograph Extension E
 92 	{  0x2CEB0,  0x2EBE0, "CJK UNIFIED IDEOGRAPH-%04X" }, -- CJK Ideograph Extension F
 93 	-- CJK Compatibility Ideographs Supplement (Supplementary Ideographic Plane)
 94 	{  0x2F800,  0x2FA1D, "CJK COMPATIBILITY IDEOGRAPH-%04X" },
 95 	{  0xE0100,  0xE01EF, function (codepoint) -- Variation Selectors Supplement
 96 		return ("VARIATION SELECTOR-%d"):format(codepoint - 0xE0100 + 17)
 97 	end},
 98 	{  0x30000,  0x3134A, "CJK UNIFIED IDEOGRAPH-%04X" }, -- CJK Ideograph Extension G
 99 	{  0x31350,  0x323AF, "CJK UNIFIED IDEOGRAPH-%04X" }, -- CJK Ideograph Extension H
100 	{  0x2EBF0,  0x2EE5D, "CJK UNIFIED IDEOGRAPH-%04X" }, -- CJK Ideograph Extension I
101 	{  0xF0000,  0xFFFFD, "<private-use-%04X>" }, -- Plane 15 Private Use
102 	{ 0x100000, 0x10FFFD, "<private-use-%04X>" }  -- Plane 16 Private Use
103 }
104 name_hooks.length = #name_hooks
105 
106 local name_range_cache
107 
108 local function generate_name(data, codepoint)
109 	if type(data) == "string" then
110 		return data:format(codepoint)
111 	else
112 		return data(codepoint)
113 	end
114 end
115 
116 --[[
117 -- Checks that the code point is a number and in range.
118 -- Does not check whether code point is an integer.
119 -- Not used
120 local function check_codepoint(funcName, argIdx, val)
121 	require 'libraryUtil'.checkType(funcName, argIdx, val, 'number')
122 	if codepoint < 0 or 0x10FFFF < codepoint then
123 		errorf("Codepoint %04X out of range", codepoint)
124 	end
125 end
126 --]]
127 
128 function p.is_noncharacter(codepoint)
129 	-- U+FDD0-U+FDEF and all code points ending in FFFE or FFFF are Unassigned
130 	-- (Cn) and specifically noncharacters:
131 
132 	-- https://www.unicode.org/faq/private_use.html#nonchar4
133 	return 0xFDD0 <= codepoint and (codepoint <= 0xFDEF
134 			or floor(codepoint % 0x10000) >= 0xFFFE)
135 end
136 
137 -- https://www.unicode.org/versions/Unicode11.0.0/ch04.pdf, section 4.8
138 function p.lookup_name(codepoint)
139 	if p.is_noncharacter(codepoint) then
140 		return ("<noncharacter-%04X>"):format(codepoint)
141 	end
142 
143 	if name_range_cache -- Check if previously used "name hook" applies to this code point.
144 			and codepoint >= name_range_cache[1]
145 			and codepoint <= name_range_cache[2] then
146 		return generate_name(name_range_cache[3], codepoint)
147 	end
148 	
149 	local range = binary_range_search(codepoint, name_hooks)
150 	if range then
151 		name_range_cache = range
152 		return generate_name(range[3], codepoint)
153 	end
154 
155 	local data = loader[('names/%03X'):format(codepoint / 0x1000)]
156 	
157 	if data and data[codepoint] then
158 		return data[codepoint]
159 	
160 	-- Unassigned (Cn) consists of noncharacters and reserved characters.
161 	-- The character has been established not to be a noncharacter,
162 	-- and if it were assigned, its name would already been retrieved,
163 	-- so it must be reserved.
164 	else
165 		return ("<reserved-%04X>"):format(codepoint)
166 	end
167 end
168 
169 function p.lookup_image(codepoint)
170 	local data = loader[('images/%03X'):format(codepoint / 0x1000)]
171 	
172 	if data then
173 		return data[codepoint]
174 	end
175 end
176 
177 local planes = {
178 	[ 0] = "Basic Multilingual Plane";
179 	[ 1] = "Supplementary Multilingual Plane";
180 	[ 2] = "Supplementary Ideographic Plane";
181 	[ 3] = "Tertiary Ideographic Plane";
182 	[14] = "Supplementary Special-purpose Plane";
183 	[15] = "Supplementary Private Use Area-A";
184 	[16] = "Supplementary Private Use Area-B";
185 }
186 
187 -- Load [[Module:Unicode data/blocks]] if needed and assign it to this variable.
188 local blocks
189 
190 local function block_iter(blocks, i)
191 	i = i + 1
192 	local data = blocks[i]
193 	if data then
194 		 -- Unpack doesn't work on tables loaded with mw.loadData.
195 		return i, data[1], data[2], data[3]
196 	end
197 end
198 
199 -- An ipairs-type iterator generator for the list of blocks.
200 function p.enum_blocks()
201 	local blocks = loader.blocks
202 	return block_iter, blocks, 0
203 end
204 
205 function p.lookup_plane(codepoint)
206 	local i = floor(codepoint / 0x10000)
207 	return planes[i] or ("Plane %u"):format(i)
208 end
209 
210 function p.lookup_block(codepoint)
211 	local blocks = loader.blocks
212 	local range = binary_range_search(codepoint, blocks)
213 	if range then
214 		return range[3]
215 	else
216 		return "No Block"
217 	end
218 end
219 
220 function p.get_block_info(name)
221 	for i, block in ipairs(loader.blocks) do
222 		if block[3] == name then
223 			return block
224 		end
225 	end
226 end
227 
228 function p.is_valid_pagename(pagename)
229 	local has_nonws = false
230 
231 	for cp in mw.ustring.gcodepoint(pagename) do
232 		if (cp == 0x0023) -- #
233 		or (cp == 0x005B) -- [
234 		or (cp == 0x005D) -- ]
235 		or (cp == 0x007B) -- {
236 		or (cp == 0x007C) -- |
237 		or (cp == 0x007D) -- }
238 		or (cp == 0x180E) -- MONGOLIAN VOWEL SEPARATOR
239 		or ((cp >= 0x2000) and (cp <= 0x200A)) -- spaces in General Punctuation block
240 		or (cp == 0xFFFD) -- REPLACEMENT CHARACTER
241 		then
242 			return false
243 		end
244 
245 		local printable, result = p.is_printable(cp)
246 		if not printable then
247 			return false
248 		end
249 
250 		if result ~= "space-separator" then
251 			has_nonws = true
252 		end
253 	end
254 
255 	return has_nonws
256 end
257 
258 local function manual_unpack(what, from)
259 	if what[from + 1] == nil then
260 		return what[from]
261 	end
262 	
263 	local result = {}
264 	from = from or 1
265 	for i, item in ipairs(what) do
266 		if i >= from then
267 			table.insert(result, item)
268 		end
269 	end
270 	return unpack(result)
271 end
272 
273 local function compare_ranges(range1, range2)
274 	return range1[1] < range2[1]
275 end
276 
277 -- Creates a function to look up data in a module that contains "singles" (a
278 -- code point-to-data map) and "ranges" (an array containing arrays that contain
279 -- the low and high code points of a range and the data associated with that
280 -- range).
281 -- "loader" loads and returns the "singles" and "ranges" tables.
282 -- "match_func" is passed the code point and either the data or the "dots", and
283 -- generates the final result of the function.
284 -- The varargs ("dots") describes the default data to be returned if there wasn't
285 -- a match.
286 -- In case the function is used more than once, "cache" saves ranges that have
287 -- already been found to match, or a range whose data is the default if there
288 -- was no match.
289 local function memo_lookup(data_module_subpage, match_func, ...)
290 	local dots = { ... }
291 	local cache = {}
292 	local singles, ranges
293 
294 	return function (codepoint)
295 		if not singles then
296 			local data_module = loader[data_module_subpage]
297 			singles, ranges = data_module.singles, data_module.ranges
298 		end
299 
300 		if singles[codepoint] then
301 			return match_func(codepoint, singles[codepoint])
302 		end
303 
304 		local range = binary_range_search(codepoint, cache)
305 		if range then
306 			return match_func(codepoint, manual_unpack(range, 3))
307 		end
308 		
309 		local range, index = binary_range_search(codepoint, ranges)
310 		if range then
311 			table.insert(cache, range)
312 			table.sort(cache, compare_ranges)
313 			return match_func(codepoint, manual_unpack(range, 3))
314 		end
315 		
316 		if ranges[index] then
317 			local dots_range
318 			if codepoint > ranges[index][2] then
319 				dots_range = {
320 					ranges[index][2] + 1,
321 					ranges[index + 1] and ranges[index + 1][1] - 1 or 0x10FFFF,
322 					unpack(dots)
323 				}
324 			else -- codepoint < range[index][1]
325 				dots_range = {
326 					ranges[index - 1] and ranges[index - 1][2] + 1 or 0,
327 					ranges[index][1] - 1,
328 					unpack(dots)
329 				}
330 			end
331 			table.sort(cache, compare_ranges)
332 		end
333 		
334 		return match_func(codepoint)
335 	end
336 end
337 
338 -- Get a code point's combining class value in [[Module:Unicode data/combining]],
339 -- and return whether this value is not zero. Zero is assigned as the default
340 -- if the combining class value is not found in this data module.
341 -- That is, return true if character is combining, or false if it is not.
342 -- See https://www.unicode.org/reports/tr44/#Canonical_Combining_Class_Values for
343 -- more information.
344 p.is_combining = memo_lookup(
345 	"combining",
346 	function (codepoint, combining_class)
347 		return combining_class and combining_class ~= 0 or false
348 	end,
349 	0)
350 
351 function p.add_dotted_circle(str)
352 	return (mw.ustring.gsub(str, ".",
353 		function(char)
354 			if p.is_combining(mw.ustring.codepoint(char)) then
355 				return '◌' .. char
356 			end
357 		end))
358 end
359 
360 local lookup_control = memo_lookup(
361 	"control",
362 	function (codepoint, ccc)
363 		return ccc or "assigned"
364 	end,
365 	"assigned")
366 p.lookup_control = lookup_control
367 
368 function p.is_assigned(codepoint)
369 	return lookup_control(codepoint) ~= "unassigned"
370 end
371 
372 function p.is_printable(codepoint)
373 	local result = lookup_control(codepoint)
374 	return (result == "assigned") or (result == "space-separator"), result
375 end
376 
377 function p.is_whitespace(codepoint)
378 	local result = lookup_control(codepoint)
379 	return (result == "space-separator"), result
380 end
381 
382 p.lookup_category = memo_lookup(
383 	"category",
384 	function (codepoint, category)
385 		return category
386 	end,
387 	"Cn")
388 
389 local lookup_script = memo_lookup(
390 	"scripts",
391 	function (codepoint, script_code)
392 		return script_code or 'Zzzz'
393 	end,
394 	"Zzzz")
395 p.lookup_script = lookup_script
396 
397 function p.get_best_script(str)
398 	-- Check type of argument, because mw.text.decode coerces numbers to strings!
399 	require "libraryUtil".checkType("get_best_script", 1, str, "string")
400 	
401 	-- Convert HTML character references (including named character references,
402 	-- or character entities) to characters.
403 	str = mw.text.decode(str, true)
404 	
405 	local scripts = {}
406 	for codepoint in mw.ustring.gcodepoint(str) do
407 		local script = lookup_script(codepoint)
408 		
409 		-- Ignore "Inherited", "Undetermined", or "Uncoded" scripts.
410 		if not (script == "Zyyy" or script == "Zinh" or script == "Zzzz") then
411 			scripts[script] = true
412 		end
413 	end
414 	
415 	-- If scripts does not contain two or more keys,
416 	-- return first and only key (script code) in table.
417 	if not next(scripts, next(scripts)) then
418 		return next(scripts)
419 	end -- else return majority script, or else "Zzzz"?
420 end
421 
422 function p.is_Latin(str)
423 	require "libraryUtil".checkType("get_best_script", 1, str, "string")
424 	str = mw.text.decode(str, true)
425 	
426 	-- Search for the leading bytes that introduce the UTF-8 encoding of the
427 	-- code points U+0340-U+10FFFF. If they are not found and there is at least
428 	-- one Latin-script character, the string counts as Latin, because the rest
429 	-- of the characters can only be Zyyy, Zinh, and Zzzz.
430 	-- The only scripts found below U+0370 (the first code point of the Greek
431 	-- and Coptic block) are Latn, Zyyy, Zinh, and Zzzz.
432 	-- See the codepage in the [[UTF-8]] article.
433 	if not str:find "[\205-\244]" then
434 		for codepoint in mw.ustring.gcodepoint(str) do
435 			if lookup_script(codepoint) == "Latn" then
436 				return true
437 			end
438 		end
439 	end
440 	
441 	local Latn = false
442 	local i = 0;																-- indexer for use in error messages
443 	
444 	for codepoint in mw.ustring.gcodepoint(str) do
445 		i = i + 1;																-- bump the indexer
446 		local script = lookup_script(codepoint)
447 		
448 		if script == "Latn" then
449 			Latn = true
450 		elseif not (script == "Zyyy" or script == "Zinh"
451 				or script == "Zzzz") then
452 			return false, i														-- abandon as not Latn; identify the offending character's position
453 		end
454 	end
455 	
456 	return Latn, (not Latn and i) or nil										-- when <Latn> false, return offending charactor's position as second return value; nil else
457 end
458 
459 -- Checks that a string contains only characters belonging to right-to-left
460 -- scripts, or characters of ignorable scripts.
461 function p.is_rtl(str)
462 	require "libraryUtil".checkType("get_best_script", 1, str, "string")
463 	str = mw.text.decode(str, true)
464 	
465 	-- Search for the leading bytes that introduce the UTF-8 encoding of the
466 	-- code points U+0580-U+10FFFF. If they are not found, the string can only
467 	-- have characters from a left-to-right script, because the first code point
468 	-- in a right-to-left script is U+0591, in the Hebrew block.
469 	if not str:find "[\214-\244]" then
470 		return false
471 	end
472 	
473 	local result = false
474 	local rtl = loader.scripts.rtl
475 	for codepoint in mw.ustring.gcodepoint(str) do
476 		local script = lookup_script(codepoint)
477 		
478 		if rtl[script] then
479 			result = true
480 		elseif not (script == "Zyyy" or script == "Zinh"
481 				or script == "Zzzz") then
482 			return false
483 		end
484 	end
485 	
486 	return result
487 end
488 
489 
490 --[[--------------------------< I S _ R T L _ F R A M E >------------------------------------------------------
491 
492 external entry from an {{#invoke:}} to determine if a string of text is rtl.  Strips html and html-like tags so
493 that those tags don't corrupt the is-rtl-is-not-rtl determination; this added for the cases where the rtl text
494 has <br /> tags.
495 
496 ]]
497 
498 function p.is_rtl_frame (frame)
499 	local str = frame.args[1];													-- get the string from the {{#invoke:}} frame
500 	str = str:gsub ('%b<>', '');												-- strip any html and html-like tags
501 	return p.is_rtl (str);														-- return if whatever remains rtl; false else
502 end
503 
504 
505 local function get_codepoint(args, arg)
506 	local codepoint_string = args[arg]
507 		or errorf(2, "Parameter %s is required", tostring(arg))
508 	local codepoint = tonumber(codepoint_string, 16)
509 		or errorf(2, "Parameter %s is not a code point in hexadecimal base",
510 			tostring(arg))
511 	if not (0 <= codepoint and codepoint <= 0x10FFFF) then
512 		errorf(2, "code point in parameter %s out of range", tostring(arg))
513 	end
514 	return codepoint
515 end
516 
517 local function get_func(args, arg, prefix)
518 	local suffix = args[arg]
519 		or errorf(2, "Parameter %s is required", tostring(arg))
520 	suffix = mw.text.trim(suffix)
521 	local func_name = prefix .. suffix
522 	local func = p[func_name]
523 		or errorf(2, "There is no function '%s'", func_name)
524 	return func
525 end
526 
527 -- This function allows any of the "lookup" functions to be invoked. The first
528 -- parameter is the word after "lookup_"; the second parameter is the code point
529 -- in hexadecimal base.
530 function p.lookup(frame)
531 	local func = get_func(frame.args, 1, "lookup_")
532 	local codepoint = get_codepoint(frame.args, 2)
533 	local result = func(codepoint)
534 	if func == p.lookup_name then
535 		-- Prevent code point labels such as <control-0000> from being
536 		-- interpreted as HTML tags.
537 		result = result:gsub("<", "&lt;")
538 	end
539 	return result
540 end
541 
542 function p.is(frame)
543 	local func = get_func(frame.args, 1, "is_")
544 	
545 	-- is_Latin and is_valid_pagename take strings.
546 	if func == p.is_Latin or func == p.is_valid_pagename or func == p.is_rtl then
547 		return (func(frame.args[2]))
548 	else -- The rest take code points.
549 		local codepoint = get_codepoint(frame.args, 2)
550 		return (func(codepoint)) -- Adjust to one result.
551 	end
552 end
553 
554 function p.lookup_kCantonese(codepoint)
555 	local data = loader[('Unihan/kCantonese/%02X'):format(floor(codepoint / 0x1000))]
556 	if data then
557 		return data[codepoint]
558 	end
559 end
560 
561 return p