Hi Valerie,
I have a few thoughts on the bulleted statements below:
1. The condition of level of supervision as a qualification of entrustment makes perfect sense. Like other contextual factors (high acuity, patient volume, system complexities such as loss of support staff, EMR, etc) I anticipate that a milestone level assigned will be determined based on observable behaviors and then reported in the context in which that score was given.
2. If portfolio free text fields are not translated to scores that inform the assessment of the domain addressed in the free text field, the portfolio may fall short in being useful for assessment. Portfolio text output can serve a number purposes some of which may be to aid self-directed learning (inform learner self-reflection, provide substrate for advisor/mentor discussion of learner identified goals). If free-text fields are interpreted (manually or through a systematic coding process) by a rater then that data can be used more easily. There are, of course, limits to the usefulness of self-assessment that extend beyond the issue of free-text fields and impact the validity of portfolios for assessment.
Patty