研究表明,仅通过优化链式思维(Chain-of-Thought, CoT)来解答问题,仅仅追求答案的正确性,可能会降低可读性,让答案变得难以理解。 为了解决这个问题,OpenAI提出了一种新方法,使用小模型来验证大模型生成的答案。 OpenAI这项研究的主要目的是让大多数人更容易理解和信任人工智能(AI)生成的答案,特别是在处理复杂问题时。为了达到这个目标,他们设计了一种特别的方法,叫做“证明者-验证者游戏”。 <strong>研究背景</strong> 随着机器学习系统在许多高风险领域的广泛应用,确保其输出的可靠性和可解释性变得至关重要。在许多情况下,复杂系统的输出需要通过人类或其他系统进行验证,以建立信任。例如,在司法系统中,法庭需要提供清晰且易于理解的书面意见来支持其决策。 对于大型语言模型(LLMs)而言,它们在许多任务中表现出色,但其输出的复杂性和模糊性可能会降低用户对其结果的信任。为了提升LLMs输出的可信度,研究人员提出了一个新的概念——可读性(legibility),即确保模型的推理过程和结果对人类而言是清晰且易于验证的。 <strong>具体目标包括:</strong> <ol> <li><strong>让答案更容易理解</strong>: <ul> <li>AI生成的答案有时太复杂,难以理解。研究希望通过训练AI生成更清晰、更容易理解的答案。</li> </ul> </li> <li><strong>保持准确性</strong>: <ul> <li>他们希望在不牺牲答案准确性的情况下,提高答案的可读性和理解度。</li> </ul> </li> <li><strong>验证答案的正确性</strong>: <ul> <li>设计了一种游戏,让一个强大的AI(称为“证明者”)生成答案,然后一个较弱的AI(称为“验证者”)检查这些答案是否正确。通过这种互动,确保生成的答案不仅正确,还容易理解。</li> </ul> </li> <li><strong>减少对人类的依赖</strong>: <ul> <li>研究希望减少对人类的依赖,直接通过AI的训练来生成易于验证和理解的答案。</li> </ul> </li> <li><strong>提高AI的透明度和可信度</strong>: <ul> <li>让AI生成的答案更透明、更容易验证,使人们对这些答案更有信心。</li> </ul> </li> </ol> <strong>为什么这很重要?</strong> 当我们使用AI解决复杂问题,比如数学题或者科学问题时,我们希望答案不仅是正确的,还要容易理解。如果答案太复杂,即使它是对的,人们也很难信任和使用。通过这个研究,OpenAI希望让AI生成的答案更清晰明了,方便人们使用和信任,从而让AI在更多领域发挥更大的作用。 <h3><img class="aligncenter size-full wp-image-11437" src="https://img.xiaohu.ai/2024/07/legibility_blog_hero-scaled.webp" alt="" width="2560" height="1440" />研究方法</h3> OpenAI在这项研究中采用了“证明者-验证者游戏”的方法来提高语言模型输出的可读性和可验证性。具体方法包括以下几个步骤: <ol> <li><strong>模型选择与初始化</strong>: <ul> <li>选择了来自GPT-4家族的强模型(证明者)和弱模型(验证者),两者在预训练计算量上有大约3个数量级的差异。</li> <li>这两个模型经过了一定程度的监督训练,使用人类书写的数学推导,但没有进行广泛的后续训练。</li> <li>要求强模型生成的解决方案能被一个更弱的模型(verifier)容易地验证。</li> </ul> </li> <li><strong>多轮交替训练</strong>: <ul> <li><strong>生成多种解决方案</strong>:首先从证明者模型生成多个数学问题的解决方案,其中一些由于模型的自然错误而不正确。</li> <li><strong>验证者训练</strong>:训练弱验证者模型预测由证明者生成的解决方案的正确性。</li> <li><strong>证明者训练</strong>:在两种角色下对强证明者模型进行训练: <ul> <li><strong>帮助型角色</strong>:奖励验证者评分高的正确解决方案。</li> <li><strong>狡猾型角色</strong>:奖励验证者评分高的错误解决方案。</li> </ul> </li> </ul> </li> <li><strong>循环训练过程</strong>: <ul> <li>在初始阶段,狡猾型证明者能够生成让验证者信服的错误解决方案。</li> <li>随着每轮训练中加入这些错误解决方案进行验证者训练,验证者逐渐提高其判断能力,狡猾型证明者的策略逐渐失效。</li> <li>训练结束时,狡猾型证明者无法再生成让验证者信服的错误解决方案,证明验证者的判别能力得到了显著提升。</li> </ul> </li> <li><strong>优化可读性</strong>: <ul> <li>优化小学数学问题的链式推理,使其易于较弱模型验证,并研究这是否使得人类更容易理解。</li> <li>这种训练程序受到证明者-验证者游戏框架的启发,旨在鼓励学习代理以可验证的方式解决决策问题。</li> </ul> </li> <li><strong>验证和测试</strong>: <ul> <li>在训练和测试过程中,通过让验证者和证明者不断交替训练和评估,确保最终的解决方案既正确又易于理解和验证。</li> </ul> </li> </ol> <strong><img class="aligncenter size-full wp-image-11440" src="https://img.xiaohu.ai/2024/07/Jietu20240718-130217@2x.jpg" alt="" width="2214" height="882" />关键步骤总结</strong> <ul> <li><strong>模型选择</strong>:选用强证明者和弱验证者模型。</li> <li><strong>交替训练</strong>:多轮交替训练证明者和验证者。</li> <li><strong>优化方法</strong>:通过优化推理过程提高可读性。</li> <li><strong>验证测试</strong>:不断通过验证者和证明者的互动改进模型性能和可读性。</li> </ul> <h4><strong>举例通俗易懂解释其具体方法</strong></h4> <ol> <li><strong>选择两种AI</strong>: <ul> <li><strong>强AI(证明者)</strong>:这个AI非常聪明,可以生成复杂的答案。</li> <li><strong>弱AI(验证者)</strong>:这个AI相对简单,它的任务是检查强AI生成的答案是否正确。</li> </ul> </li> <li><strong>生成和检查答案</strong>: <ul> <li><strong>生成答案</strong>:首先,强AI生成一些答案,包括一些复杂的问题,比如数学题。</li> <li><strong>检查答案</strong>:接着,弱AI检查这些答案,看它们是否正确。</li> </ul> </li> <li><strong>交替训练</strong>: <ul> <li>研究团队让这两个AI反复进行一个类似游戏的过程。在这个过程中: <ul> <li><strong>强AI生成答案</strong>:强AI尽量生成正确的答案,有时候还会故意生成一些看起来正确但其实错误的答案。</li> <li><strong>弱AI检查答案</strong>:弱AI尝试识别这些答案是否正确,并学习如何更好地判断答案的正确性。</li> </ul> </li> </ul> </li> <li><strong>不断改进</strong>: <ul> <li>在每一轮训练中,强AI和弱AI都在学习和改进。强AI学习如何生成既正确又容易理解的答案,而弱AI学习如何更准确地识别正确答案。</li> <li>经过多轮训练后,强AI逐渐变得更擅长生成易于理解的正确答案,而弱AI也变得更擅长检查这些答案。</li> </ul> </li> <li><strong>优化可读性</strong>: <ul> <li>研究人员还特别关注如何让答案的表达更清晰。他们让强AI生成的答案不仅正确,还要表达得非常清楚,使人和其他AI都能容易理解和验证。</li> </ul> </li> </ol> <div class="flex-1 overflow-hidden"> <div class="h-full"> <div class="react-scroll-to-bottom--css-wodvy-79elbk h-full"> <div class="react-scroll-to-bottom--css-wodvy-1n7m0yu"> <div class="flex flex-col text-sm md:pb-9"> <div class="w-full text-token-text-primary" dir="auto" data-testid="conversation-turn-17" data-scroll-anchor="true"> <div class="text-base py-[18px] px-3 md:px-4 m-auto md:px-5 lg:px-1 xl:px-5"> <div class="mx-auto flex flex-1 gap-4 text-base md:gap-5 lg:gap-6 md:max-w-3xl lg:max-w-[40rem] xl:max-w-[48rem]"> <div class="group/conversation-turn relative flex w-full min-w-0 flex-col agent-turn"> <div class="flex-col gap-1 md:gap-3"> <div class="flex flex-grow flex-col max-w-full"> <div class="min-h-[20px] text-message flex w-full flex-col items-end gap-2 whitespace-pre-wrap break-words [.text-message+&]:mt-5 overflow-x-auto" dir="auto" data-message-author-role="assistant" data-message-id="4ddd8aa7-1db4-4613-8b7f-b33db2a2a006"> <div class="flex w-full flex-col gap-1 empty:hidden first:pt-[3px]"> <div class="markdown prose w-full break-words dark:prose-invert light"> <h4>举个例子</h4> 假设有一道数学题:“<span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">肖</span></span></span><span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">娜</span></span></span>的爸爸是<span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">肖</span></span></span><span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">娜</span></span></span>年龄的五倍。莎娜的年龄是<span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">阿丽娅</span></span></span>的三倍。如果<span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">阿丽娅</span></span></span>3岁,<span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">肖</span></span></span><span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">娜 </span></span></span>的爸爸多大?” <ol> <li><strong>强AI生成答案</strong>:强AI可能会生成两个答案,一个是正确的45岁,一个是错误的36岁。</li> <li><strong>弱AI检查答案</strong>:弱AI会尝试判断哪个答案是对的,并学习如何更好地做出判断。</li> <li><strong>改进过程</strong>:通过反复练习,强AI学会了更好地解释每一步骤,使答案更清楚。例如,“<span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">阿丽娅</span></span></span>3岁,所以莎娜是3×3=9<em>岁,她的爸爸是9×5=45岁</em>。”</li> </ol> 最终,通过这种方法,研究团队希望AI不仅能给出正确的答案,还能以一种清晰明了的方式解释答案,让人们更容易理解和信任这些答案。 </div> <h3><img class="aligncenter size-full wp-image-11435" src="https://img.xiaohu.ai/2024/07/下载-49-scaled.jpeg" alt="" width="1254" height="2560" />研究结果</h3> OpenAI的研究显示,通过“证明者-验证者游戏”的方法,可以让AI生成的答案更容易理解,同时保持较高的准确性。以下是研究的主要结果,用简单的语言解释: <ol> <li><strong>更容易理解的答案</strong>: <ul> <li>研究发现,通过训练AI生成易于其他较弱AI验证的答案,这些答案也变得更容易让人类理解。</li> <li>例如,在解决数学问题时,AI不仅给出正确答案,还会详细解释每一步骤,使答案更清晰。</li> </ul> </li> <li><strong>减少错误评估</strong>: <ul> <li>当人类评估高度优化但难以理解的答案时,错误率几乎是评估更易理解答案的两倍。</li> <li>优化后的答案不仅正确,还更容易被快速理解和评估,减少了人类评估员的错误。</li> </ul> </li> <li><strong>三种有用的AI角色</strong>: <ul> <li><strong>鲁棒验证者</strong>:可以有效区分正确和错误的答案,即使这些答案被设计得具有误导性。</li> <li><strong>帮助型证明者</strong>:生成对人类来说易于理解的答案,减少了人类评估员的错误。</li> <li><strong>狡猾型证明者</strong>:生成微妙的错误答案,这些答案最初会迷惑人类评估员,但在多轮训练后,验证者学会了识别这些错误。</li> </ul> </li> <li><strong>训练方法的有效性</strong>: <ul> <li>使用证明者-验证者游戏的训练方法,平衡了答案的准确性和可读性,使得AI生成的答案既正确又易于理解。</li> <li>这种方法减少了对人类示范和判断的依赖,增强了AI生成答案的透明度和可信度。</li> </ul> </li> </ol> <strong><img class="aligncenter size-full wp-image-11436" src="https://img.xiaohu.ai/2024/07/Chart_Desktop_Light.svg" alt="" width="2160" height="1080" />举个例子</strong> 假设有一道数学题:“<span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">肖</span></span></span><span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">娜</span></span></span>的爸爸是莎娜年龄的五倍。<span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">肖</span></span></span><span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">娜</span></span></span>的年龄是<span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">阿丽娅</span></span></span>的三倍。如果<span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">阿丽娅</span></span></span>3岁,<span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">肖</span></span></span><span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">娜</span></span></span>的爸爸多大?” 通过“证明者-验证者游戏”的训练,AI生成了以下清晰的解释: <ol> <li><span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">阿丽娅</span></span></span>3岁。</li> <li><span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">肖</span></span></span><span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">娜</span></span></span>是<span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">阿丽娅</span></span></span>年龄的三倍,所以莎娜是3×3=9岁。</li> <li><span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">肖</span></span></span><span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">娜</span></span></span>的爸爸是莎娜年龄的五倍,所以<span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">肖</span></span></span><span class="notranslate immersive-translate-target-wrapper" lang="zh-CN" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-translation-theme-none immersive-translate-target-translation-block-wrapper-theme-none immersive-translate-target-translation-block-wrapper" data-immersive-translate-translation-element-mark="1"><span class="notranslate immersive-translate-target-inner immersive-translate-target-translation-theme-none-inner" data-immersive-translate-translation-element-mark="1">娜</span></span></span>的爸爸是9×5=45岁。</li> </ol> 这个答案不仅正确,而且每一步都解释得很清楚,让人很容易理解。 <img class="aligncenter size-full wp-image-11438" src="https://img.xiaohu.ai/2024/07/下载-50-scaled.jpeg" alt="" width="866" height="2560" />原文:<a href="https://openai.com/index/prover-verifier-games-improve-legibility/" target="_blank" rel="noopener">https://openai.com/index/prover-verifier-games-improve-legibility/</a> 论文:<a href="https://cdn.openai.com/prover-verifier-games-improve-legibility-of-llm-outputs/legibility.pdf" target="_blank" rel="noopener">https://cdn.openai.com/prover-verifier-games-improve-legibility-of-llm-outputs/legibility.pdf</a> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div>