Decoding script invitation

[xumx]

Is there a way to send in requests for the decoding script?

I understand the nature of the challenges surrounding reddit toxicity, we just want to try it out privately, and test different prompts.

[dreasysnail]

Hi Xumx,

Thanks for your interests. Due to the company policy we cannot share the decoding script freely. We are still work on a channel where researchers can register and apply for the demo access. Please stay tuned and we will update you once it is ready.

[qywu]

The decoding script is not really that hard to implement. But I am not fully sure if the inputs are exactly the same with the original implementation due to the weird tokenization implemented in this repo.
Anyway, I have attached the Colab link to whoever is interested:
https://colab.research.google.com/drive/1PslHE4Rl4RqSa20s7HEp0ZKITBir6ezE

[andreamad8]

Qingyang Wu (@qywu) I was about to post something similar :)

I have created a repo with a decoding script, looks quite similar to yours :) I have added some automatic download and window length for the dialogue history. If you have sometimes check it out, and let me know if is there something to improve.

I test it a bit yesterday and the responses are actually very good. I have tried some of the input reported in the repo and I can reproduce some of the responses too.

This is the link to the repo: https://github.com/andreamad8/DialoGPT2-Interact

I hope this is helpful

Andrea

[qywu]

Andrea Madotto (@andreamad8) Cool. Have you noticed the weird tokenization for words? It seems that they feed tokenized sentences to GPT2, but it is not necessary.

[andreamad8]

Oh, I have noticed that honestly, :) I went straight with the hugging face implementation. Which line are you referring too?
I noticed instead that by using this: tokenizer.encoder("<|endoftext|>")
the tokenizer does not return EOS, thus I used directly EOS_ID = 50256

[qywu]

DialoGPT/reddit_extractor/src/reddit.py

Lines 114 to 115 in ef531a9

	tokenizer = TweetTokenizer(preserve_case=True)
	txt = ' ' + ' '.join(tokenizer.tokenize(txt)) + ' '

I am not referring to your code, but theirs. For GPT-2, there is no need to tokenize words first. So it doesn't generate sentences like: "Hello , how are you doing ? "

[andreamad8]

Oh, I didn't check that file, but you are right, no need, GPT tokenizer does the job already.

Maybe open another issue? but good to know.

Andrea

[LHolten]

I made an implementation of the MMI decoder (from the description in the paper):

https://github.com/LHolten/DialoGTP-MMI-decoder

It features unlimited chat length and usage as a discord bot.

[drogozhang]

Qingyang Wu (@qywu) Great Job! If we want to batch the input_ids, what should we pad? the 0 results are terrible.

[yangjianxin1]

Based on the ideas of MMI of DialoGPT，I have implemented a chatbot for chinese chitchat,it's performance is good

https://github.com/yangjianxin1/GPT2-chitchat

[jsedoc]

Andrea Madotto (@andreamad8) can you post the responses that you got? ... and how you got them. We can't seem to match the ones reported.

[yangjianxin1]

the dialogue generated by chatbot is listed as follow:
Sample 1:
user :你好啊
chatbot:你好，请问您是？
user :我是你的主人啊
chatbot:你是狗吗？
user :你才是狗
chatbot:你不是狗我是你的小主人
user :你这个大笨蛋
chatbot:你大笨蛋，你是想挨打了
user :你全家都是大笨蛋
chatbot:我是你爸爸
user :算了，不跟你吵了，你在干嘛呢
chatbot:我在上班啊，你还没下班呢啊你
user :我周末要做作业啊
chatbot:不是才交了吗，早晚做完啊
user :好多作业啊，还没做完呢
chatbot:我在抄，我妈让我周五交，还有事
user :你就不怕被你妈妈骂吗
chatbot:不怕
user :你周末有去哪里玩吗
chatbot:没时间啊，忙得很啊
user :忙着干嘛呢
chatbot:不知道啊，我周末没有作业，我想睡觉，然后还得找人

Sample 2:
user :在吗
chatbot:在
user :我不开心
chatbot:怎么啦
user :最近好累啊
chatbot:是挺累的
user :作业好多啊
chatbot:还好还好
user :我觉得我快要挂科了
chatbot:我也是啊，但是我觉得我快熬出头了
user :不说了，我还有好多作业没写
chatbot:我也是啊好多啊
user :那先这样，下次再聊，再见了
chatbot:好哒