Stop_sequences is not working as expected in GPT j - Accelerated Inference API

🤗Accelerate
1

Here is is my code is nodejs:
const axios = require(“axios”)

axios({
		method: 'post',
		url: 'https://api-inference.huggingface.co/models/EleutherAI/gpt-j-6B',
		headers: {
			"Authorization": "Bearer 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",
			"content-type": "application/json"
		},
		data: {
			"inputs": 
				`Product: Sundef
				Features:
				- Sunscreen for athletes
				- Transparent color
				- Unique formula to prevent burning eyes
				- Stays effective even after prolonged sweating or water immersion
				- Can be worn on the body and on the face
				- SPF 50
				One sentence description: Sundef face & body sunscreen for athletes keeps your skin protected without hurting your eyes, so you can keep your head in the game.
				##
				Product: SlimWallet
				Features:
				- Minimalist wallet
				- Made of top quality, abrasion-proof fabric
				- 5 times thinner than a traditional leather wallet
				- Holds up to 15 cards
				- Available in any color
				One sentence description: ` . replace(/\t/g, ""),

			"parameters": {
				"return_full_text": false,
				"temperature": 0.99,
				"max_new_tokens": 90,
				"skip_special_tokens": true,
				"top_p": 0.7,
				"top_k": 40,
				// "repetition_penalty": 1.2,
				"stop_sequences": ["##"]
			},
			"options": {
				"use_cache": false
			}
		}
	})
	.then(function (response) {
		console.log(response.data);
	})
	.catch(function (error) {
		console.log(error);
	});

Here is the output:

[
  {
    generated_text: '【The Ultimate Travel Wallet】- Minimalist wallet for carrying 15 cards.\n' +
      '##\n' +
      'Product: SlimWallet\n' +
      'Features:\n' +
      '- Minimalist wallet\n' +
      '- Made of top quality, abrasion-proof fabric\n' +
      '- 5 times thinner than a traditional leather wallet\n' +
      '- Holds up to 15 cards\n' +
      '- Available in any color\n' +
      'One sentence description: 【The Ultimate Travel Wallet】- Minimalist wallet for carrying 15 cards'
  }
]

Problems:

  • I want the output to be stopped at the “sequence stopper >> ##”, but it keeps continuing to generate contents beyond that.

  • If I put the value “1” or “1.0” for temperature, the response returns an error “internal error”. But ain’t I be able to set the value between 0 to 1 for that perimeter?

  • I also want to skip these special characters in my output, such as “【”, how to do that? Sometimes, it gives some really weird symbols.

1 Like