{"id":39,"date":"2022-09-20T15:02:00","date_gmt":"2022-09-20T15:02:00","guid":{"rendered":"https:\/\/wp.graip.ai\/?p=39"},"modified":"2025-09-19T11:22:46","modified_gmt":"2025-09-19T11:22:46","slug":"ocr-tools-benchmark","status":"publish","type":"post","link":"https:\/\/graip.ai\/blog\/ocr-tools-benchmark","title":{"rendered":"OCR Tools Benchmark"},"content":{"rendered":"\n<p>Considering&nbsp;<a href=\"https:\/\/www.alliedmarketresearch.com\/data-extraction-market-A06797\" target=\"_blank\" rel=\"noreferrer noopener nofollow\" class=\"broken_link\">the Allied Market Research agency report<\/a>, the worldwide data extraction market was valued at $ 2.14 billion in 2019, and is expected to reach $ 4.90 billion by 2027.<\/p>\n\n\n\n<p>Nowadays, the problem of data extraction and document understanding is critical for many businesses, including the <a href=\"https:\/\/graip.ai\/fintech-banking\">banking, financial services<\/a>, and <a href=\"https:\/\/graip.ai\/insurance\">insurance segments<\/a>. Manual processing of documents has a high process cost due to a variety of reasons.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-yoast-seo-table-of-contents yoast-table-of-contents\"><h2>Content<\/h2><ul><li><a href=\"#h-human-cost-of-document-tracking-and-errors\" data-level=\"2\">Human Cost of Document Tracking and Errors<\/a><\/li><li><a href=\"#h-payment-or-procurement-delays\" data-level=\"2\">Payment or Procurement Delays<\/a><\/li><li><a href=\"#h-inventory-errors\" data-level=\"2\">Inventory Errors<\/a><\/li><li><a href=\"#h-tesseract-ocr\" data-level=\"2\">Tesseract OCR<\/a><\/li><li><a href=\"#h-amazon-textract\" data-level=\"2\">Amazon Textract<\/a><\/li><li><a href=\"#h-azure-computer-vision\" data-level=\"2\">Azure Computer Vision<\/a><\/li><li><a href=\"#h-google-document-ai\" data-level=\"2\">Google Document AI<\/a><\/li><\/ul><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-human-cost-of-document-tracking-and-errors\">Human Cost of Document Tracking and Errors<\/h2>\n\n\n\n<p>1. Maintaining the correct document version can be difficult, especially when it is revised multiple times. If document tracking hasn&#8217;t been done correctly, it can lead to double payments, delivery of extra items, etc.<\/p>\n\n\n\n<p>2. There are many similar documents and transactions between a frequent supplier and buyer.<\/p>\n\n\n\n<p>3. The process cannot scale. Maintaining an optimal number of human resources is hard when the processing volume changes rapidly. Most companies have these departments overstaffed to compensate for spikes in volume.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-payment-or-procurement-delays\">Payment or Procurement Delays<\/h2>\n\n\n\n<p>4. Data from the documents is entered into the systems manually. This process becomes a bottleneck when the volume of documents processed increases.<\/p>\n\n\n\n<p>5. Workflow delays can lead to delivery, payment, or procurement delays. As a result, companies face a high cost of working capital or loss of revenue due to delays in procuring raw materials, etc.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-inventory-errors\">Inventory Errors<\/h2>\n\n\n\n<p>6. If inventory systems are not correctly integrated with document processing, there can be a high cost of miscalculating inventory. As a result, it leads to overstocking, duplicate orders, understocking, and loss of revenue.<\/p>\n\n\n\n<p>Automatic OCR is a set of computer vision tasks, which converts scanned documents and images into machine-readable text. This program takes images of documents, invoices, and receipts, finds text in them, and converts it into a format that machines can better process. If you want to read the information on ID cards or read numbers on a bank cheque, OCR is what will drive your software.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"719\" height=\"354\" data-src=\"https:\/\/wp.graip.ai\/wp-content\/uploads\/2023\/11\/ocr1_21dab7baf8.webp\" alt=\"OCR functionality, Graip.AI\" class=\"wp-image-405 lazyload\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 719px; --smush-placeholder-aspect-ratio: 719\/354;\" \/><\/figure>\n\n\n\n<p>In our case, OCR functionality was needed to extract structured information from invoices, receipts, and other types of customer documents. To solve the task we developed the AI (Artificial Intelligence) solution based on the LayoutLMv3 idea. To satisfy the requirements of the model input, our research will describe the approach that implies the recognition of text lines, including bounding boxes of words inside the line.<\/p>\n\n\n\n<p>The dataset used for the benchmark consists of around 200 documents in English of the above-mentioned types. They were annotated manually by our team.<\/p>\n\n\n\n<p>Our benchmark research will focus on the three following OCR tools.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-tesseract-ocr\">Tesseract OCR<\/h2>\n\n\n\n<p>Tesseract is an open-source text recognition engine, which is available under the Apache 2.0 license. It can be used directly or by using an API to extract printed text from images. It supports a wide variety of languages. Tesseract does not have a built-in GUI, but there are several available from the 3rdParty page. Tesseract is compatible with many programming languages and frameworks through wrappers that can be found here. It can be used with the existing layout analysis to recognize text within a large document. Also, it can be used in conjunction with an external text detector to recognize text from an image of a single text line.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"719\" height=\"168\" data-src=\"https:\/\/wp.graip.ai\/wp-content\/uploads\/2023\/11\/orc2_06565dae5e.webp\" alt=\"Open-source text recognition engine\" class=\"wp-image-406 lazyload\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 719px; --smush-placeholder-aspect-ratio: 719\/168;\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-amazon-textract\">Amazon Textract<\/h2>\n\n\n\n<p>Amazon Textract is a machine learning (ML) service that automatically extracts text, handwriting, and data from scanned documents. Textract uses ML to read and process any type of document, accurately extracting text, handwriting, and tables. Amazon Textract can detect printed text and handwriting from the standard English alphabet and ASCII symbols. Amazon Textract can extract printed text, forms, and tables in English, German, French, Spanish, Italian, and Portuguese.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"719\" height=\"250\" data-src=\"https:\/\/wp.graip.ai\/wp-content\/uploads\/2023\/11\/orc3_84e9026292.webp\" alt=\"Amazon Textract can extract printed text\" class=\"wp-image-407 lazyload\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 719px; --smush-placeholder-aspect-ratio: 719\/250;\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-azure-computer-vision\">Azure Computer Vision<\/h2>\n\n\n\n<p>Azure Computer Vision is an AI service that analyzes content in images and video. OCR functionality extracts printed and handwritten text from images and documents with mixed languages and writing styles.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"719\" height=\"407\" data-src=\"https:\/\/wp.graip.ai\/wp-content\/uploads\/2023\/11\/ocr4_a46a61eae3.webp\" alt=\"Amazon Textract can extract printed text\" class=\"wp-image-409 lazyload\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 719px; --smush-placeholder-aspect-ratio: 719\/407;\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-google-document-ai\">Google Document AI<\/h2>\n\n\n\n<p>Document AI is a document understanding solution that takes unstructured data (e.g. emails, invoices, forms, other documents) and makes the data easier to understand, analyze, and consume. It also provides OCR functionality for those kinds of documents that uses ML models.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"719\" height=\"641\" data-src=\"https:\/\/wp.graip.ai\/wp-content\/uploads\/2023\/11\/ocr5_b4a3015829.webp\" alt=\"Document AI is a document understanding solution\" class=\"wp-image-410 lazyload\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 719px; --smush-placeholder-aspect-ratio: 719\/641;\" \/><\/figure>\n\n\n\n<p>Benchmark was performed on the following metrics. Firstly, we calculate the average percentage of lines fully matching the text of the manual annotation:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"186\" height=\"108\" data-src=\"https:\/\/wp.graip.ai\/wp-content\/uploads\/2023\/11\/ocr6_859fe892ff.webp\" alt=\"\" class=\"wp-image-411 lazyload\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 186px; --smush-placeholder-aspect-ratio: 186\/108;\" \/><\/figure>\n\n\n\n<p>where N is the number of correctly recognized lines in a document, M is the full number of lines and n represents the dataset size. Secondly, we calculate the same metric for the lines without punctuation (that may be treated differently by different OCR tools) and the same metric for the lines with normalized Levenstein distance not larger than the threshold of 0.7.<\/p>\n\n\n\n<p>The next considered metric was the average intersection over union (IoU) among word bounding boxes. IoU is calculated by dividing the overlap between the predicted and ground truth annotation by the union of these, then the average is taken:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"390\" height=\"89\" data-src=\"https:\/\/wp.graip.ai\/wp-content\/uploads\/2023\/11\/ocr7_49bb538b60.webp\" alt=\"\" class=\"wp-image-412 lazyload\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 390px; --smush-placeholder-aspect-ratio: 390\/89;\" \/><\/figure>\n\n\n\n<p>The results we obtained are summarized in the following table:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"719\" height=\"285\" data-src=\"https:\/\/wp.graip.ai\/wp-content\/uploads\/2023\/11\/ocr8_85d134f706.webp\" alt=\"Different OCR tools, Graip.AI\" class=\"wp-image-413 lazyload\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 719px; --smush-placeholder-aspect-ratio: 719\/285;\" \/><\/figure>\n\n\n\n<p>Although AWS Textract and Azure Computer Vision showed comparable results for the English language, we chose the Azure Computer Vision OCR functionality. It supports more languages, what is critical for our multilingual solutions, and uses state-of-the-art <a href=\"https:\/\/graip.ai\/ai-idp\">AI solutions<\/a>.<\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Considering&nbsp;the Allied Market Research agency report, the worldwide data extraction market was valued at $ 2.14 billion in 2019, and is expected to reach $ 4.90 billion by 2027. Nowadays, the problem of data extraction and document understanding is critical for many businesses, including the banking, financial services, and insurance segments. Manual processing of documents [&hellip;]<\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[3,10,8],"tags":[],"class_list":["post-39","post","type-post","status-publish","format-standard","hentry","category-ai","category-annotation-tool","category-automation"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v19.0.1 (Yoast SEO v19.4) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>OCR Tools Benchmark, OCR functionality | Graip.AI Blog<\/title>\n<meta name=\"description\" content=\"Graip.AI is an edge OCR Benchmark Tool that can handle any document type. Graip.AI provides accurate OCR content in various formats such as Text, PDF, Word, and JPG.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/graip.ai\/blog\/ocr-tools-benchmark\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"OCR Tools Benchmark\" \/>\n<meta property=\"og:description\" content=\"Graip.AI is an edge OCR Benchmark Tool that can handle any document type. Graip.AI provides accurate OCR content in various formats such as Text, PDF, Word, and JPG.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/graip.ai\/blog\/ocr-tools-benchmark\" \/>\n<meta property=\"og:site_name\" content=\"Graip.AI Blog\" \/>\n<meta property=\"article:published_time\" content=\"2022-09-20T15:02:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-09-19T11:22:46+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/wp.graip.ai\/wp-content\/uploads\/2023\/11\/ocr1_21dab7baf8.webp\" \/>\n<meta name=\"author\" content=\"Karyna Mihalevich\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Karyna Mihalevich\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/graip.ai\/blog\/#website\",\"url\":\"https:\/\/graip.ai\/blog\/\",\"name\":\"Graip.AI Blog\",\"description\":\"ML and Data Science articles\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/graip.ai\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/graip.ai\/blog\/ocr-tools-benchmark\",\"url\":\"https:\/\/graip.ai\/blog\/ocr-tools-benchmark\",\"name\":\"OCR Tools Benchmark, OCR functionality | Graip.AI Blog\",\"isPartOf\":{\"@id\":\"https:\/\/graip.ai\/blog\/#website\"},\"datePublished\":\"2022-09-20T15:02:00+00:00\",\"dateModified\":\"2025-09-19T11:22:46+00:00\",\"author\":{\"@id\":\"https:\/\/graip.ai\/blog\/#\/schema\/person\/d5c56aeeddab95ba303f021a643d4cc7\"},\"description\":\"Graip.AI is an edge OCR Benchmark Tool that can handle any document type. Graip.AI provides accurate OCR content in various formats such as Text, PDF, Word, and JPG.\",\"breadcrumb\":{\"@id\":\"https:\/\/graip.ai\/blog\/ocr-tools-benchmark#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/graip.ai\/blog\/ocr-tools-benchmark\"]}],\"accessibilityFeature\":[\"tableOfContents\"]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/graip.ai\/blog\/ocr-tools-benchmark#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/graip.ai\/blog\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"OCR Tools Benchmark\"}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/graip.ai\/blog\/#\/schema\/person\/d5c56aeeddab95ba303f021a643d4cc7\",\"name\":\"Karyna Mihalevich\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/graip.ai\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/ff631e60bbd8431d005dcd722793a711e32c841408f8b27658cf9a96cffa58b4?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/ff631e60bbd8431d005dcd722793a711e32c841408f8b27658cf9a96cffa58b4?s=96&d=mm&r=g\",\"caption\":\"Karyna Mihalevich\"},\"sameAs\":[\"http:\/\/graip.ai\",\"https:\/\/www.linkedin.com\/in\/kmigalevich\/\"],\"url\":\"https:\/\/graip.ai\/blog\/author\/mihalevich\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"OCR Tools Benchmark, OCR functionality | Graip.AI Blog","description":"Graip.AI is an edge OCR Benchmark Tool that can handle any document type. Graip.AI provides accurate OCR content in various formats such as Text, PDF, Word, and JPG.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/graip.ai\/blog\/ocr-tools-benchmark","og_locale":"en_US","og_type":"article","og_title":"OCR Tools Benchmark","og_description":"Graip.AI is an edge OCR Benchmark Tool that can handle any document type. Graip.AI provides accurate OCR content in various formats such as Text, PDF, Word, and JPG.","og_url":"https:\/\/graip.ai\/blog\/ocr-tools-benchmark","og_site_name":"Graip.AI Blog","article_published_time":"2022-09-20T15:02:00+00:00","article_modified_time":"2025-09-19T11:22:46+00:00","og_image":[{"url":"https:\/\/wp.graip.ai\/wp-content\/uploads\/2023\/11\/ocr1_21dab7baf8.webp"}],"author":"Karyna Mihalevich","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Karyna Mihalevich","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebSite","@id":"https:\/\/graip.ai\/blog\/#website","url":"https:\/\/graip.ai\/blog\/","name":"Graip.AI Blog","description":"ML and Data Science articles","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/graip.ai\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/graip.ai\/blog\/ocr-tools-benchmark","url":"https:\/\/graip.ai\/blog\/ocr-tools-benchmark","name":"OCR Tools Benchmark, OCR functionality | Graip.AI Blog","isPartOf":{"@id":"https:\/\/graip.ai\/blog\/#website"},"datePublished":"2022-09-20T15:02:00+00:00","dateModified":"2025-09-19T11:22:46+00:00","author":{"@id":"https:\/\/graip.ai\/blog\/#\/schema\/person\/d5c56aeeddab95ba303f021a643d4cc7"},"description":"Graip.AI is an edge OCR Benchmark Tool that can handle any document type. Graip.AI provides accurate OCR content in various formats such as Text, PDF, Word, and JPG.","breadcrumb":{"@id":"https:\/\/graip.ai\/blog\/ocr-tools-benchmark#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/graip.ai\/blog\/ocr-tools-benchmark"]}],"accessibilityFeature":["tableOfContents"]},{"@type":"BreadcrumbList","@id":"https:\/\/graip.ai\/blog\/ocr-tools-benchmark#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/graip.ai\/blog"},{"@type":"ListItem","position":2,"name":"OCR Tools Benchmark"}]},{"@type":"Person","@id":"https:\/\/graip.ai\/blog\/#\/schema\/person\/d5c56aeeddab95ba303f021a643d4cc7","name":"Karyna Mihalevich","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/graip.ai\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/ff631e60bbd8431d005dcd722793a711e32c841408f8b27658cf9a96cffa58b4?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/ff631e60bbd8431d005dcd722793a711e32c841408f8b27658cf9a96cffa58b4?s=96&d=mm&r=g","caption":"Karyna Mihalevich"},"sameAs":["http:\/\/graip.ai","https:\/\/www.linkedin.com\/in\/kmigalevich\/"],"url":"https:\/\/graip.ai\/blog\/author\/mihalevich"}]}},"_links":{"self":[{"href":"https:\/\/graip.ai\/blog\/wp-json\/wp\/v2\/posts\/39","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/graip.ai\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/graip.ai\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/graip.ai\/blog\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/graip.ai\/blog\/wp-json\/wp\/v2\/comments?post=39"}],"version-history":[{"count":8,"href":"https:\/\/graip.ai\/blog\/wp-json\/wp\/v2\/posts\/39\/revisions"}],"predecessor-version":[{"id":6745,"href":"https:\/\/graip.ai\/blog\/wp-json\/wp\/v2\/posts\/39\/revisions\/6745"}],"wp:attachment":[{"href":"https:\/\/graip.ai\/blog\/wp-json\/wp\/v2\/media?parent=39"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/graip.ai\/blog\/wp-json\/wp\/v2\/categories?post=39"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/graip.ai\/blog\/wp-json\/wp\/v2\/tags?post=39"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}