OpenAI Gym 入门教程

草莓干 发表于 2020-12-25 15:49:31

本帖最后由草莓干于 2020-12-25 15:49 编辑 <section style="width:100%;margin:0px auto;padding:0px;" data-width="100%"><section style="width: 100%;border:1px solid rgb(173,218,204);" data-width="100%"><section style="padding:3px;"><section data-bgless="spin" data-bglessp="120" data-bgopacity="10%" style="color: rgb(63,62,63);background:rgb(247,251,249);"><section class="135brush" style="font-size: 14px;text-align: justify;letter-spacing: 1px;line-height: 25px;padding:0.5em;">除了试图直接去建立一个可以模拟成人大脑的程序之外， 为什么不试图建立一个可以模拟小孩大脑的程序呢?如果它接受适当的教育，就会获得成人的大脑。 — 阿兰·图灵</section></section></section></section></section><h2 style="line-height: 1.75em;"></h2> <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;font-size: 18px;box-sizing: border-box;"><section style="display: inline-block;border-radius: 6px;line-height: 1.6em;background-color: rgb(169, 218, 228);box-sizing: border-box;"><section style="border-radius: 6px 0px 0px 6px;display: inline-block;vertical-align: top;width: 1.6em;height: 1.6em;color: rgb(255, 255, 255);background-color: rgb(248, 205, 208);box-sizing: border-box;">01</section><section style="padding-right: 10px;padding-left: 10px;display: inline-block;vertical-align: top;color: rgb(255, 255, 255);font-size: 15px;line-height: 2;letter-spacing: 2px;box-sizing: border-box;">介绍</section></section></section>强化学习 (Reinforcement learning) 是机器学习的一个子领域用于制定决策和运动自由度控制。强化学习主要研究在复杂未知的环境中，智体(agent)实现某个目标。强化学习最引人入胜的两个特点是：1. 强化学习非常通用，可以用来解决需要作出一些列决策的所有问题： 例如，训练机器人跑步和弹跳，制定商品价格和库存管理，玩 Atari 游戏和棋盘游戏等等。2. 强化学习已经可以在许多复杂的环境中取得较好的实验结果：例如 Deep RL 的 Alpha Go等。 Gym 是一个研究和开发强化学习相关算法的仿真平台。<ul class=" list-paddingleft-2" style="list-style-type: disc;"><li>无需智体先验知识；</li><li>兼容常见的数值运算库如 TensorFlow、Theano 等</li></ul> Gym 的一个最小例子 CartPole-v0 运行效果 至此，第一个 Hello world 就算正式地跑起来了！ <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;font-size: 18px;box-sizing: border-box;"><section style="display: inline-block;border-radius: 6px;line-height: 1.6em;background-color: rgb(169, 218, 228);box-sizing: border-box;"><section style="border-radius: 6px 0px 0px 6px;display: inline-block;vertical-align: top;width: 1.6em;height: 1.6em;color: rgb(255, 255, 255);background-color: rgb(248, 205, 208);box-sizing: border-box;">02</section><section style="padding-right: 10px;padding-left: 10px;display: inline-block;vertical-align: top;color: rgb(255, 255, 255);font-size: 15px;line-height: 2;letter-spacing: 2px;box-sizing: border-box;">观测(Observations) </section></section></section>在第一个小栗子中，使用了 env.step() 函数来对每一步进行仿真，在 Gym 中，env.step() 会返回 4 个参数： <ul class=" list-paddingleft-2" style="list-style-type: disc;"><li>观测 Observation (Object)：当前 step 执行后，环境的观测(类型为对象)。例如，从相机获取的像素点，机器人各个关节的角度或棋盘游戏当前的状态等；</li><li>奖励 Reward (Float): 执行上一步动作(action)后，智体(agent)获得的奖励(浮点类型)，不同的环境中奖励值变化范围也不相同，但是强化学习的目标就是使得总奖励值最大；</li><li>完成 Done (Boolen): 表示是否需要将环境重置 env.reset。大多数情况下，当 Done 为True 时，就表明当前回合(episode)或者试验(tial)结束。例如当机器人摔倒或者掉出台面，就应当终止当前回合进行重置(reset);</li><li>信息 Info (Dict): 针对调试过程的诊断信息。在标准的智体仿真评估当中不会使用到这个 info，具体用到的时候再说。</li></ul> 总结来说，这就是一个强化学习的基本流程，在每个时间点上，智体执行 action，环境返回上一次 action 的观测和奖励，用图表示为 <span style="margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0px; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;">在 Gym 仿真中，每一次回合开始，需要先执行 <span style="margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0px; background: rgb(246, 246, 246); box-sizing: border-box !important; overflow-wrap: break-word !important;">reset()<span style="margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0px; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;"> 函数，返回初始观测信息，然后根据标志位 <span style="margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0px; background: rgb(246, 246, 246); box-sizing: border-box !important; overflow-wrap: break-word !important;">done<span style="margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0px; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;"> 的状态，来决定是否进行下一次回合。代码表示为<span style="font-size: 14px; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0px; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;"><span style="font-size: 14px; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0px; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;"> <span style="background: rgb(255, 255, 255); margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0px; font-size: 14px; font-family: 微软雅黑, Microsoft YaHei; box-sizing: border-box !important; overflow-wrap: break-word !important;">仿真截图如下：<span style="background: rgb(255, 255, 255); margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0px; font-size: 14px; font-family: 微软雅黑, Microsoft YaHei; box-sizing: border-box !important; overflow-wrap: break-word !important;"><span style="background: rgb(255, 255, 255); margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0px; font-size: 14px; font-family: 微软雅黑, Microsoft YaHei; box-sizing: border-box !important; overflow-wrap: break-word !important;"> <span style="background: rgb(255, 255, 255); margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0px; font-size: 14px; font-family: 微软雅黑, Microsoft YaHei; box-sizing: border-box !important; overflow-wrap: break-word !important;"><span style="margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 34px; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0px; font-size: 15px; background: rgb(255, 255, 255);">每次 <span style="margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 34px; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0px; font-size: 15px; background: rgb(246, 246, 246);">action<span style="margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 34px; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0px; font-size: 15px; background: rgb(255, 255, 255);"> 前，将上一次 <span style="margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 34px; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0px; font-size: 15px; background: rgb(246, 246, 246);">observation<span style="margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 34px; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0px; font-size: 15px; background: rgb(255, 255, 255);"> 打印，可以得到打印日志如下 <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;font-size: 18px;box-sizing: border-box;"><section style="display: inline-block;border-radius: 6px;line-height: 1.6em;background-color: rgb(169, 218, 228);box-sizing: border-box;"><section style="border-radius: 6px 0px 0px 6px;display: inline-block;vertical-align: top;width: 1.6em;height: 1.6em;color: rgb(255, 255, 255);background-color: rgb(248, 205, 208);box-sizing: border-box;">03</section><section style="padding-right: 10px;padding-left: 10px;display: inline-block;vertical-align: top;color: rgb(255, 255, 255);font-size: 15px;line-height: 2;letter-spacing: 2px;box-sizing: border-box;">空间（Spaces） </section></section></section>在前面的两个小栗子中，每次执行的动作(action)都是从环境动作空间中随机进行选取的，但是这些动作 (action) 是什么?在 Gym 的仿真环境中，有运动间 action_space 和观测空间observation_space 两个指标，程序中被定义为 Space类型，用于描述有效的运动和观测的格式和范围。下面是一个代码示例: 从程序运行结果来看<span style="background: rgb(255, 255, 255); text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; box-sizing: border-box !important; overflow-wrap: break-word !important;">· <span style="text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; background: rgb(246, 246, 246); box-sizing: border-box !important; overflow-wrap: break-word !important;">action_space<span style="background: rgb(255, 255, 255); text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; box-sizing: border-box !important; overflow-wrap: break-word !important;"> 是一个离散 <span style="text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; background: rgb(246, 246, 246); box-sizing: border-box !important; overflow-wrap: break-word !important;">Discrete<span style="background: rgb(255, 255, 255); text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; box-sizing: border-box !important; overflow-wrap: break-word !important;"> 类型，从 <span style="background: rgb(255, 255, 255); color: rgb(51, 51, 51); text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; letter-spacing: 0pt; font-size: 11pt; box-sizing: border-box !important; overflow-wrap: break-word !important;">discrete.py <span style="background: rgb(255, 255, 255); text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; font-family: 微软雅黑; box-sizing: border-box !important; overflow-wrap: break-word !important;">源码可知，范围是一个<span style="text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; background: rgb(246, 246, 246); box-sizing: border-box !important; overflow-wrap: break-word !important;">{0,1,...,n-1}<span style="background: rgb(255, 255, 255); text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; box-sizing: border-box !important; overflow-wrap: break-word !important;"> 长度为 <span style="text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; background: rgb(246, 246, 246); box-sizing: border-box !important; overflow-wrap: break-word !important;">n<span style="background: rgb(255, 255, 255); text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; box-sizing: border-box !important; overflow-wrap: break-word !important;"> 的非负整数集合，在 <span style="text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; background: rgb(246, 246, 246); box-sizing: border-box !important; overflow-wrap: break-word !important;">CartPole-v0<span style="background: rgb(255, 255, 255); text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; box-sizing: border-box !important; overflow-wrap: break-word !important;"> 例子中，动作空间表示为<span style="text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; background: rgb(246, 246, 246); box-sizing: border-box !important; overflow-wrap: break-word !important;">{0,1}<span style="background: rgb(255, 255, 255); text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; font-family: 微软雅黑; box-sizing: border-box !important; overflow-wrap: break-word !important;">。<span style="background: rgb(255, 255, 255); text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; box-sizing: border-box !important; overflow-wrap: break-word !important;">· <span style="text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; background: rgb(246, 246, 246); box-sizing: border-box !important; overflow-wrap: break-word !important;">observation_space<span style="background: rgb(255, 255, 255); text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; box-sizing: border-box !important; overflow-wrap: break-word !important;"> 是一个 <span style="text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; background: rgb(246, 246, 246); box-sizing: border-box !important; overflow-wrap: break-word !important;">Box<span style="background: rgb(255, 255, 255); text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; box-sizing: border-box !important; overflow-wrap: break-word !important;"> 类型，从 <span style="background: rgb(255, 255, 255); color: rgb(51, 51, 51); text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; letter-spacing: 0pt; font-size: 11pt; box-sizing: border-box !important; overflow-wrap: break-word !important;">box.py <span style="background: rgb(255, 255, 255); text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; box-sizing: border-box !important; overflow-wrap: break-word !important;">源码可知，表示一个 <span style="text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; background: rgb(246, 246, 246); box-sizing: border-box !important; overflow-wrap: break-word !important;">n<span style="background: rgb(255, 255, 255); text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; box-sizing: border-box !important; overflow-wrap: break-word !important;"> 维的盒子，所以在上一节打印出来的 <span style="text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; background: rgb(246, 246, 246); box-sizing: border-box !important; overflow-wrap: break-word !important;">observation<span style="background: rgb(255, 255, 255); text-align: justify; text-indent: 22.5pt; margin: 0px; padding: 0px; max-width: 100%; font-family: 微软雅黑; color: rgb(26, 26, 26); letter-spacing: 0pt; font-size: 11pt; box-sizing: border-box !important; overflow-wrap: break-word !important;"> 是一个长度为 4 的数组。数组中的每个元素都具有上下界。 <span style="font-family: 微软雅黑, Microsoft YaHei; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;">利用运动空间和观测空间的定义和范围，可以将代码写得更加通用。在许多仿真环境中，<span style="font-family: 微软雅黑, Microsoft YaHei; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background: rgb(246, 246, 246); box-sizing: border-box !important; overflow-wrap: break-word !important;">Box<span style="font-family: 微软雅黑, Microsoft YaHei; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;"> 和<span style="font-family: 微软雅黑, Microsoft YaHei; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background: rgb(246, 246, 246); box-sizing: border-box !important; overflow-wrap: break-word !important;">Discrete<span style="font-family: 微软雅黑, Microsoft YaHei; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;"> 是最常见的空间描述，在智体每次执行动作时，都属于这些空间范围内，代码示例为：<span style="font-size: 14px; font-family: 微软雅黑, Microsoft YaHei; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;"><span style="font-size: 14px; font-family: 微软雅黑, Microsoft YaHei; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;"> <span style="margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; font-size: 14px; font-family: 微软雅黑, Microsoft YaHei; box-sizing: border-box !important; overflow-wrap: break-word !important;">在 CartPole-v0 栗子中，运动只能选择左和右，分别用 {0,1} 表示。<span style="font-size: 14px; font-family: 微软雅黑, Microsoft YaHei; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;"> <span style="font-size: 14px; font-family: 微软雅黑, Microsoft YaHei; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;"> <span style="font-size: 14px; font-family: 微软雅黑, Microsoft YaHei; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;"><section style="margin-top: 10px;margin-bottom: 10px;text-align: center;font-size: 18px;box-sizing: border-box;"><section style="display: inline-block;border-radius: 6px;line-height: 1.6em;background-color: rgb(169, 218, 228);box-sizing: border-box;"><section style="border-radius: 6px 0px 0px 6px;display: inline-block;vertical-align: top;width: 1.6em;height: 1.6em;color: rgb(255, 255, 255);background-color: rgb(248, 205, 208);box-sizing: border-box;">04</section><section style="padding-right: 10px;padding-left: 10px;display: inline-block;vertical-align: top;color: rgb(255, 255, 255);font-size: 15px;line-height: 2;letter-spacing: 2px;box-sizing: border-box;">Gym中可用的环境 </section></section></section><span style="font-size: 14px; font-family: 微软雅黑, Microsoft YaHei; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;">Gym 中从简单到复杂，包含了许多经典的仿真环境和各种数据，其中包括：<ul class=" list-paddingleft-2" style="list-style-type: disc;"><li>经典控制和文字游戏：经典的强化学习示例，方便入门；</li><li>算法：从例子中学习强化学习的相关算法，在 Gym 的仿真算法中，由易到难方便新手入坑；</li><li>雅达利游戏：利用强化学习来玩雅达利的游戏。Gym 中集成了对强化学习有着重要影响的Arcade Learning Environment，并且方便用户安装；</li><li>2D 和 3D 的机器人：这个是我一直很感兴趣的一部分，在 Gym 中控制机器人进行仿真。需要利用第三方的物理引擎如 MuJoCo 。</li></ul> <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;font-size: 18px;box-sizing: border-box;"><section style="display: inline-block;border-radius: 6px;line-height: 1.6em;background-color: rgb(169, 218, 228);box-sizing: border-box;"><section style="border-radius: 6px 0px 0px 6px;display: inline-block;vertical-align: top;width: 1.6em;height: 1.6em;color: rgb(255, 255, 255);background-color: rgb(248, 205, 208);box-sizing: border-box;">05</section><section style="padding-right: 10px;padding-left: 10px;display: inline-block;vertical-align: top;color: rgb(255, 255, 255);font-size: 15px;line-height: 2;letter-spacing: 2px;box-sizing: border-box;">注册表 </section></section></section>Gym 是一个包含各种各样强化学习仿真环境的大集合，并且封装成通用的接口暴露给用户，查看所有环境的代码如下： <span style="font-family: 微软雅黑, Microsoft YaHei; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;">Gym 支持将用户制作的环境写入到注册表中，需要执行 <span style="font-family: 微软雅黑, Microsoft YaHei; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background: rgb(246, 246, 246); box-sizing: border-box !important; overflow-wrap: break-word !important;">gym.make()<span style="font-family: 微软雅黑, Microsoft YaHei; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;"> 和在启动时注册<span style="font-family: 微软雅黑, Microsoft YaHei; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background: rgb(246, 246, 246); box-sizing: border-box !important; overflow-wrap: break-word !important;">register<span style="font-family: 微软雅黑, Microsoft YaHei; margin: 0px; padding: 0px; max-width: 100%; text-align: justify; text-indent: 30px; color: rgb(26, 26, 26); letter-spacing: 0pt; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial; box-sizing: border-box !important; overflow-wrap: break-word !important;">，例如： <section style="margin-top: 10px;margin-bottom: 10px;text-align: center;font-size: 18px;box-sizing: border-box;"><section style="display: inline-block;border-radius: 6px;line-height: 1.6em;background-color: rgb(169, 218, 228);box-sizing: border-box;"><section style="border-radius: 6px 0px 0px 6px;display: inline-block;vertical-align: top;width: 1.6em;height: 1.6em;color: rgb(255, 255, 255);background-color: rgb(248, 205, 208);box-sizing: border-box;">06</section><section style="padding-right: 10px;padding-left: 10px;display: inline-block;vertical-align: top;color: rgb(255, 255, 255);font-size: 15px;line-height: 2;letter-spacing: 2px;box-sizing: border-box;">结语 </section></section></section>emmmm ... 第一篇强化学习入坑笔记写完，大多是从官方文档看过来的加上了一点点自己的理解，建议文档这东西还是直接看官方的吧，原汁原味。 参考链接如下： https://gym.openai.com/docs/https://nndl.github.io/https://github.com/openai/gym/blob/master/gym/spaces/discrete.pyhttps://github.com/openai/gym/blob/master/gym/spaces/box.pyhttps://gym.openai.com/envs/#classic_control <link rel="stylesheet" href="//bbs.lejurobot.com/source/plugin/wcn_editor/public/wcn_editor_fit.css?v134_kKx" id="wcn_editor_css"/>

页: [1]

乐聚社区's Archiver

OpenAI Gym 入门教程