It's called Improv (short for Improvisation), the act of going off-roading from the set script and making up entire chunks of dialogue or characterization.
Improvisation is closely related to ad-libbing, a short (usually one or two lines) deviation, but here the connotations are that it happens frequently, if not actually the entire acting method.
Actors and actresses are regularly ad libbing, improvising or going off-script while reciting their lines. Sometimes the directors hate it – other times they love it.
Occasionally the improved lines become immortalized as some of the most memorable in cinema history. Check out ScreenRant.com's 32 Greatest Unscripted Movie Scenes.
But when an entire production is based around improvisation, you're probably looking at a variant of Theatresports, a competition between teams of improvisers that use theatre games and/or improv scenes.