Add stream output support

#21
This comment has been hidden

Add streaming support for text generation

  • Implemented streaming functionality for real-time text output.
  • Added _decode_stream method to handle text streaming.
  • Updated chat method to support streaming mode.
  • Adjusted code to process and yield text in chunks for better responsiveness.

This update enhances the user experience by allowing incremental text generation and display.

JamePeng2023 changed pull request status to open
Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment