> For the complete documentation index, see [llms.txt](https://x.ancorasir.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://x.ancorasir.com/me336/2020-spring/me336-collaborative-robot-learning.md).

# \[ME336] Collaborative Robot Learning

2020 Spring

## Course Description

This course introduces how robots learn to perceive the physical world, act in it and make reliable plans. It connects collaborative robotics and robotic perception with artificial intelligence, machine learning and deep learning. Topics include computer vision, 3D perception, intelligent agents, game playing, Markov decision processes, neural networks, regularization, optimization and modern network architectures.

## Teaching Goals

* Understand the technical challenges of human–robot collaboration and robot learning.
* Develop robot programs that integrate perception, decision-making and learning.
* Apply principles from mechanical engineering, mathematics and computer science to robotic systems.
* Communicate engineering decisions, implementation methods and results through code, presentations and written reports.

## Course Instructor & Teaching Team

**Instructor:** Song Chaoyang\
**Department:** Mechanical and Energy Engineering, Southern University of Science and Technology (SUSTech)\
**Email:** <songcy@sustech.edu.cn>\
**Term:** 2020 Spring\
**Credits:** 3\
**Teaching language:** English

## Prerequisite

ME306 Fundamentals of Robotics.

## Grading Policy

The final grade consists of **classroom participation (10%)** and **individual course projects (90%)**.

| Component               | Weight | Requirements                                                                             |
| ----------------------- | ------ | ---------------------------------------------------------------------------------------- |
| Classroom participation | 10%    | One point per attendance check over 15 weeks, normalized to the participation component. |
| Assignment project 1    | 20%    | Code submission 10%; video report 10%.                                                   |
| Assignment project 2    | 20%    | Code submission 10%; video report 10%.                                                   |
| Assignment project 3    | 20%    | Code submission 10%; video report 10%.                                                   |
| Final project           | 30%    | Project report 10%; video report 10%; presentation slides 5%; code submission 5%.        |

Each project is completed individually. Code is assessed for successful execution, completion of the assigned task, comments, structure, readability and reuse. Video reports explain the technical background, implementation, demonstration and discussion. The final written report follows the IEEE conference format and presents background, methods, experiments and results with appropriate references. Presentation slides are assessed for content, clarity, organization and explanation. Submissions should follow the assigned format, destination and timing requirements.

[Download the 2020 assessment criteria (DOCX)](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FwzpKyxzjW4xZ2E7ooHZx%2F200603-%E3%80%8CME336%E5%8D%8F%E4%BD%9C%E6%9C%BA%E5%99%A8%E4%BA%BA%E5%AD%A6%E4%B9%A0%E3%80%8D%E8%AF%BE%E7%A8%8B%E8%AF%84%E5%88%86%E6%A0%87%E5%87%86.docx?alt=media\&token=6c760b50-4a8c-4448-9080-1d4456521ec2)

## Teaching Schedule

* [Lecture 01 — Course Introduction](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FCXKYFmaJBPaYZduZyDP3%2FLecture%2001-Course%20Introduction.pdf?alt=media\&token=1c860aa1-39dd-469a-b269-29b995cbc082)
* [Lecture 02 — Computer Vision](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FoTw7d89uUSeffU6fO9gD%2FLecture%2002-Computer%20Vision.pdf?alt=media\&token=5a135331-3705-4ead-9205-ff6c486e0cd3)
* [Lecture 03 — 3D Perception](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FFMdwRv2eKdlnBnnb4PxF%2FLecture%2003-3D%20Perception.pdf?alt=media\&token=3df33f87-ec89-4e8e-9e45-6a624c982643)
* [Lecture 04 — Rational Agents](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2F5BAnUG3VICe9u4s3pLRY%2FLecture%2004-Rational%20Agents.pdf?alt=media\&token=37c09e4b-bc2b-4d4c-afd7-4859f3aab7a4)
* [Lecture 05 — Game Playing](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2F19L2PvfinvejOdmyRHyJ%2FLecture%2005-Game%20Playing.pdf?alt=media\&token=673b9f8f-904f-4219-8f64-ae1e20ecb277)
* [Lecture 06 — Markov Decision Processes](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FKIOfwIA1XeGgBJ9yZPtu%2FLecture%2006-Markov%20Decision%20Processes.pdf?alt=media\&token=20a1ea12-2c92-42c3-9be8-ef8f782708ca)
* [Lecture 07 — Machine Learning-Part 1](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FNZmNmGpjJDQwagV3gn2m%2FLecture%2007-Machine%20Learning-Part%201.pdf?alt=media\&token=3ea7e19e-f70e-4cc4-ba1b-e15b8319d71f)
* [Lecture 07 — Machine Learning-Part 2](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FqW5eRVXdTHIIBfuYHgMn%2FLecture%2007-Machine%20Learning-Part%202.pdf?alt=media\&token=cdc20bb9-6d45-493b-b6e5-d38193ce1040)
* [Lecture 08 — Deep Network](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FXx7irARJpmRIxQj7yJyu%2FLecture%2008-Deep%20Network.pdf?alt=media\&token=7c8d319f-ff8f-4664-bca6-658e39aaf3df)
* [Lecture 09 — Network Tuning](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FCk5HIbNvw8R8PV86GfX1%2FLecture%2009-Network%20Tuning.pdf?alt=media\&token=b5eda125-7934-4b8b-b4f7-5c3a9edd2d5f)
* [Lecture 10 — Modern Architectures](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FtKiaB58uLARAxPBWNFC6%2FLecture%2010-Modern%20Architectures.pdf?alt=media\&token=312db4e4-d67b-406d-8fa7-5faf4ea11436)

## Course Materials & Resources

### Course Documents

* [Course specification (DOCX)](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FaMYw6ddUyviNfKtI9gok%2F200603-%E3%80%8CME336%E5%8D%8F%E4%BD%9C%E6%9C%BA%E5%99%A8%E4%BA%BA%E5%AD%A6%E4%B9%A0%E3%80%8D%E6%95%99%E5%AD%A6%E5%A4%A7%E7%BA%B2.docx?alt=media\&token=13244e88-482b-44ae-a69f-d9e549846933)

The grading policy above follows the separate 2020 assessment criteria.

### Computing Skills Workshops

The Missing Semester of Your CS Education materials:

* [Lecture 1: Course Overview + The Shell (2020)](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2Fz61QgWDgZoNfJfMcb3ll%2FLecture%201_%20Course%20Overview%20%2B%20The%20Shell%20\(2020\).pdf?alt=media\&token=fb6e9ef7-0397-47c9-a618-316b264a8735)
* [Lecture 2: Shell Tools and Scripting (2020)](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2Fn0gos0H1d6U8GoxlvQUm%2FLecture%202_%20Shell%20Tools%20and%20Scripting%20\(2020\).pdf?alt=media\&token=6d16fb23-0542-49d0-a072-013fcd0a85c4)
* [Lecture 3: Editors (vim) (2020)](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FQbSKqqvmm19aSWHQy4ta%2FLecture%203_%20Editors%20\(vim\)%20\(2020\).pdf?alt=media\&token=8f411883-3e15-413a-85f3-168df2321670)
* [Lecture 4: Data Wrangling (2020)](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FFyUiDLXftKRLLPdnJMkz%2FLecture%204_%20Data%20Wrangling%20\(2020\).pdf?alt=media\&token=2001a2ff-794a-4b24-abf0-8cfd02b8d477)
* [Lecture 5: Command-line Environment (2020)](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FkMXXoAawndYb1Pp46t2S%2FLecture%205_%20Command-line%20Environment%20\(2020\).pdf?alt=media\&token=0242e710-61f5-48ff-8fcb-5a7e8bd6e79f)
* [Lecture 6: Version Control (Git) (2020)](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FnYsKWtBOoElFKn1nMIMu%2FLecture%206_%20Version%20Control%20\(Git\)%20\(2020\).pdf?alt=media\&token=fdc74edd-0930-4c13-9fe2-266141b45e67)
* [Lecture 7: Debugging and Profiling (2020)](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FL4OL3VzThJQq6Rq10qMg%2FLecture%207_%20Debugging%20and%20Profiling%20\(2020\).pdf?alt=media\&token=0c6292a2-a84b-40ce-9466-81d6f24bb707)
* [Lecture 8: Metaprogramming (2020)](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FuP4XhjH8EatZ4kcz7a5Y%2FLecture%208_%20Metaprogramming%20\(2020\).pdf?alt=media\&token=2051096e-c89a-48be-99fa-027dd4fc67ab)
* [Lecture 9: Security and Cryptography (2020)](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2F2JJwPteRzPI0d4zqgaWQ%2FLecture%209_%20Security%20and%20Cryptography%20\(2020\).pdf?alt=media\&token=4f18fa17-82bd-40dd-a341-804c56adedf5)
* [Lecture 10: Potpourri (2020)](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FRHMCjlMYAwgP0C1GhL0e%2FLecture%2010_%20Potpourri%20\(2020\).pdf?alt=media\&token=08eb8d0b-8252-45b0-b23b-297024ead178)
* [Lecture 11: Q\&A (2020)](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2FGKUxFOj0XMq0yPu5Y4e9%2FLecture%2011_%20Q%26A%20\(2020\).pdf?alt=media\&token=60d886ca-bd52-4739-8c05-01c73a0a4f6d)

### Guest Lecture

* [Guest lecture slides](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6SM5cuYf8Y8R0G09kZSv%2Fuploads%2F1PB3xtDv7o0GT4Nkahaj%2FME336-Guest%20Lecture.pdf?alt=media\&token=34a06f85-d896-4f38-ab07-7b82d8901766)

### Textbooks & References

**Required**

* Richard M. Murray, Zexiang Li and S. Shankar Sastry, *A Mathematical Introduction to Robotic Manipulation*.

**Supplementary**

* Lentin Joseph, *Mastering ROS for Robotics Programming*.
* John Craig, *Introduction to Robotics: Mechanics and Control*, 4th edition.
* Peter Corke, *Robotics, Vision and Control: Fundamental Algorithms in MATLAB*.
* Stuart Russell and Peter Norvig, *Artificial Intelligence: A Modern Approach*, 3rd edition.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://x.ancorasir.com/me336/2020-spring/me336-collaborative-robot-learning.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.