About the recaps, right now that is a separate flow. You'll have to create the event and then in hte Events List modal, use the ··· dropdown menu to get the "Add recap" option.
Apologies for the poor experience of event management in the slack settings. We've realized that the limitations of Slack's UI building blocks really prevent us from creating custom-fit UI we need for complex flows like this. So it is on our immediate roadmap to be moving these settings over into the admin Studio where we can tailor fit the UX.
is there a way within the slack app itself to see the cache of past events?
Yes at the top of the Events List modal you can filter by Past events: https://docs.tightknit.ai/events/getting-started#events-list-panel