Yes, that would be possible - though it would be a suite of modules, some which manage courses, some faculty, some students, attach persons to a course with given role (faculty, student, other roles as needed), terms, academic years, grading ... you are looking at an enterprise application which needs lots of investigation regarding requirements.
It can be done inside of DNN, yes, as well as outside. To leverage DNN security roles would be helpful but not sufficient as you may have folks who are student in one course and teaching faculty in another, or student in some and grad assistant in another. Therefore you'd have to have security roles within your module suite which allow such things.
Using DNN would be a good choice, however, in my opinion, because of other added value for features provided by the framework, core and third-party modules for content in addition to the rather demanding/daunting application you are considering.
On the other hand, if you are looking ONLY to develop the kind of functionality you mention, and not take advantage of other DNN-based features, then there's probably not a lot of point in building this inside of DNN. Won't be significantly helpful to do so in that case.
aside: Interesting - you have zero posts, and I'm responding to one of them ... :)