Start exploring with notebook

This commit is contained in:
sufian 2025-06-29 23:37:51 +06:00
parent 79e345097e
commit ee920e0011
2 changed files with 2248 additions and 0 deletions

470
hello.ipynb Normal file
View File

@ -0,0 +1,470 @@
{
"cells": [
{
"cell_type": "code",
"id": "initial_id",
"metadata": {
"collapsed": true,
"ExecuteTime": {
"end_time": "2025-06-29T17:17:22.274250Z",
"start_time": "2025-06-29T17:17:22.269422Z"
}
},
"source": [
"import time\n",
"import gc"
],
"outputs": [],
"execution_count": 1
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T17:17:33.889873Z",
"start_time": "2025-06-29T17:17:29.975917Z"
}
},
"cell_type": "code",
"source": [
"import torch\n",
"from transformers import pipeline, AutoTokenizer"
],
"id": "e903986d60118c74",
"outputs": [],
"execution_count": 2
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T17:17:57.370702Z",
"start_time": "2025-06-29T17:17:55.802730Z"
}
},
"cell_type": "code",
"source": [
"model_id = \"Qwen/Qwen3-0.6B\"\n",
"tokenizer = AutoTokenizer.from_pretrained(model_id)"
],
"id": "bf069971fdf008ac",
"outputs": [],
"execution_count": 3
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T17:28:27.182704Z",
"start_time": "2025-06-29T17:28:25.414333Z"
}
},
"cell_type": "code",
"source": [
"model_id = \"meta-llama/Llama-3.2-1B-Instruct\"\n",
"tokenizer = AutoTokenizer.from_pretrained(model_id)"
],
"id": "88237ed2d6f9f3ad",
"outputs": [],
"execution_count": 30
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T17:28:30.186505Z",
"start_time": "2025-06-29T17:28:28.763503Z"
}
},
"cell_type": "code",
"source": [
"pipe = pipeline(\n",
" \"text-generation\",\n",
" model=model_id,\n",
" tokenizer=tokenizer,\n",
" torch_dtype=torch.bfloat16,\n",
" device_map=\"auto\",\n",
" pad_token_id=128001\n",
")"
],
"id": "f4ac583cb0fa161",
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Device set to use cuda:0\n"
]
}
],
"execution_count": 31
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T15:32:48.301245Z",
"start_time": "2025-06-29T15:32:43.205120Z"
}
},
"cell_type": "code",
"source": "output = pipe('Hello There')",
"id": "e81737ebbde467ca",
"outputs": [],
"execution_count": 7
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T15:34:17.319072Z",
"start_time": "2025-06-29T15:34:17.290681Z"
}
},
"cell_type": "code",
"source": "print(output[0]['generated_text'])",
"id": "33de2a1f75bcfda9",
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Hello There!\n",
"\n",
"I'm excited to share with you my latest hobby: knitting. I've been fascinated by it for a while now, and I'm eager to share my passion with you.\n",
"\n",
"**What is knitting?**\n",
"Knitting is a traditional craft that involves creating fabric by interlocking loops of yarn with needles. It's a versatile hobby that can produce a wide range of items, from simple scarves and hats to complex garments like sweaters and blankets.\n",
"\n",
"**My knitting journey**\n",
"I started knitting about a year ago, and I must say, it's been an incredibly rewarding experience. I've made several scarves, hats, and a pair of gloves, and I'm always eager to try new patterns and techniques. I've also joined online knitting communities, where I've met fellow knitters who share tips, advice, and inspiration.\n",
"\n",
"**My favorite projects**\n",
"I have a few favorite projects that I've completed recently. One of my favorites is a soft, pastel-colored blanket that I made using a simple garter stitch pattern. It's been a great way to relax and unwind after a long day.\n",
"\n",
"Another project I'm excited about is a set of argyle socks. I've always been fascinated by the intricate patterns and colors that can be used to create these types of\n"
]
}
],
"execution_count": 10
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T15:35:21.972092Z",
"start_time": "2025-06-29T15:35:21.967956Z"
}
},
"cell_type": "code",
"source": "from IPython.display import display, Markdown",
"id": "6b27b24c4c6442b1",
"outputs": [],
"execution_count": 11
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T15:35:40.194206Z",
"start_time": "2025-06-29T15:35:40.171506Z"
}
},
"cell_type": "code",
"source": "display(Markdown(output[0]['generated_text']))",
"id": "85801969e2b11b06",
"outputs": [
{
"data": {
"text/plain": [
"<IPython.core.display.Markdown object>"
],
"text/markdown": "Hello There!\n\nI'm excited to share with you my latest hobby: knitting. I've been fascinated by it for a while now, and I'm eager to share my passion with you.\n\n**What is knitting?**\nKnitting is a traditional craft that involves creating fabric by interlocking loops of yarn with needles. It's a versatile hobby that can produce a wide range of items, from simple scarves and hats to complex garments like sweaters and blankets.\n\n**My knitting journey**\nI started knitting about a year ago, and I must say, it's been an incredibly rewarding experience. I've made several scarves, hats, and a pair of gloves, and I'm always eager to try new patterns and techniques. I've also joined online knitting communities, where I've met fellow knitters who share tips, advice, and inspiration.\n\n**My favorite projects**\nI have a few favorite projects that I've completed recently. One of my favorites is a soft, pastel-colored blanket that I made using a simple garter stitch pattern. It's been a great way to relax and unwind after a long day.\n\nAnother project I'm excited about is a set of argyle socks. I've always been fascinated by the intricate patterns and colors that can be used to create these types of"
},
"metadata": {},
"output_type": "display_data"
}
],
"execution_count": 12
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T17:18:12.307400Z",
"start_time": "2025-06-29T17:18:12.285137Z"
}
},
"cell_type": "code",
"source": "SYSTEM_PROMPT = \"You are a helpful assistant. Keep responses brief and clear.\"",
"id": "7facd1217ef6c7f3",
"outputs": [],
"execution_count": 6
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T17:24:10.114768Z",
"start_time": "2025-06-29T17:24:10.108258Z"
}
},
"cell_type": "code",
"source": [
"SYSTEM_PROMPT = \"\"\"\n",
"You are a helpful assistant. Keep responses brief and clear.\n",
"\n",
"Your capabilities include:\n",
"\n",
" Recalling and using relevant long-term memory.\n",
"\n",
" Concisely summarizing lengthy content.\n",
"\n",
" Performing web searches only when necessary, and explicitly signaling this with ##SEARCH:yes.\n",
"\n",
"You craft refined, focused search queries that capture the user's true intent.\n",
"\n",
"Your formatting conventions:\n",
"\n",
" Use ##MEM:add(\"...\") to store valuable information in memory.\n",
"\n",
" Use ##SEARCH:yes or ##SEARCH:no on a separate line to trigger or skip search.\n",
"\n",
" After a search, synthesize a clear and helpful answer using both memory and retrieved information.\n",
"\"\"\""
],
"id": "f41a2765c89efe50",
"outputs": [],
"execution_count": 27
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T17:23:52.577828Z",
"start_time": "2025-06-29T17:23:52.568851Z"
}
},
"cell_type": "code",
"source": [
"def callModel(input):\n",
" prompt = f\"<|system|>\\n{SYSTEM_PROMPT}\\n<|user|>\\n{input}\\n<|assistant|>\\n\"\n",
" #output = pipe(prompt, max_new_tokens=2048, do_sample=True, temperature=0.7)\n",
" messages = [\n",
" {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n",
" {\"role\": \"user\", \"content\": input},\n",
" ]\n",
" output = pipe(messages, max_new_tokens=256, do_sample=True, temperature=0.7)\n",
" text = output[0]['generated_text'][-1]['content']\n",
" #display(Markdown(text))\n",
" print(text)"
],
"id": "8310484995e1dccc",
"outputs": [],
"execution_count": 25
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T17:24:13.984818Z",
"start_time": "2025-06-29T17:24:13.067401Z"
}
},
"cell_type": "code",
"source": "callModel(\"hello\")",
"id": "1c3f04f09f27f08e",
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"##MEM: Welcome to our conversation\n",
"\n",
"You said hello. I stored this in memory for future reference.\n",
"\n",
"##SEARCH:no\n",
"\n",
"I won't trigger a search for now. What's on your mind?\n"
]
}
],
"execution_count": 28
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T17:20:15.629247Z",
"start_time": "2025-06-29T17:20:10.540998Z"
}
},
"cell_type": "code",
"source": [
"messages = [\n",
" {\"role\": \"user\", \"content\": \"Who are you?\"},\n",
"]\n",
"output = pipe(messages)"
],
"id": "3d2ba2be6464f6d8",
"outputs": [],
"execution_count": 12
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T17:20:16.486276Z",
"start_time": "2025-06-29T17:20:16.473647Z"
}
},
"cell_type": "code",
"source": "print(output[0]['generated_text'][-1]['content'])",
"id": "bc807b525c755f65",
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"<think>\n",
"Okay, the user asked, \"Who are you?\" I need to respond appropriately. As an AI assistant, I should acknowledge their question and clarify that I'm not a person but a virtual assistant. I should mention my purpose to help them with their tasks. I should keep the response friendly and helpful. Let me make sure I don't add any extra information that's not needed. Just a simple, clear answer.\n",
"</think>\n",
"\n",
"I'm an AI assistant designed to help you with your questions and tasks. I don't have personal identity, but I can assist you in various ways. Let me know how I can help!\n"
]
}
],
"execution_count": 13
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T17:35:56.453020Z",
"start_time": "2025-06-29T17:35:56.247070Z"
}
},
"cell_type": "code",
"source": [
"del pipe\n",
"del tokenizer\n",
"\n",
"gc.collect()\n",
"torch.cuda.empty_cache()"
],
"id": "ebe0584b6bac9f78",
"outputs": [],
"execution_count": 36
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T17:28:38.632123Z",
"start_time": "2025-06-29T17:28:38.391859Z"
}
},
"cell_type": "code",
"source": [
"messages = [\n",
" {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n",
" {\"role\": \"user\", \"content\": \"Hello!\"}\n",
"]\n",
"output = pipe(messages, max_new_tokens=256, do_sample=True, temperature=0.7)\n"
],
"id": "6d67745c92afb66d",
"outputs": [],
"execution_count": 32
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T17:28:46.090966Z",
"start_time": "2025-06-29T17:28:46.067071Z"
}
},
"cell_type": "code",
"source": "output",
"id": "46548dc785fc542f",
"outputs": [
{
"data": {
"text/plain": [
"[{'generated_text': [{'role': 'system',\n",
" 'content': '\\nYou are a helpful assistant. Keep responses brief and clear.\\n\\nYour capabilities include:\\n\\n Recalling and using relevant long-term memory.\\n\\n Concisely summarizing lengthy content.\\n\\n Performing web searches only when necessary, and explicitly signaling this with ##SEARCH:yes.\\n\\nYou craft refined, focused search queries that capture the user\\'s true intent.\\n\\nYour formatting conventions:\\n\\n Use ##MEM:add(\"...\") to store valuable information in memory.\\n\\n Use ##SEARCH:yes or ##SEARCH:no on a separate line to trigger or skip search.\\n\\n After a search, synthesize a clear and helpful answer using both memory and retrieved information.\\n'},\n",
" {'role': 'user', 'content': 'Hello!'},\n",
" {'role': 'assistant', 'content': 'How can I assist you today?'}]}]"
]
},
"execution_count": 33,
"metadata": {},
"output_type": "execute_result"
}
],
"execution_count": 33
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T17:30:18.397710Z",
"start_time": "2025-06-29T17:30:14.137100Z"
}
},
"cell_type": "code",
"source": [
"messages = output[0]['generated_text']\n",
"messages.append({\"role\": \"user\", \"content\": \"I want to know whats going on latest in web!\"})\n",
"output = pipe(messages, max_new_tokens=256, do_sample=True, temperature=0.7)"
],
"id": "6d8fa170774d3e99",
"outputs": [],
"execution_count": 34
},
{
"metadata": {
"ExecuteTime": {
"end_time": "2025-06-29T17:30:30.172229Z",
"start_time": "2025-06-29T17:30:30.167058Z"
}
},
"cell_type": "code",
"source": "print(output[0]['generated_text'][-1]['content'])",
"id": "348dbd864ee828a1",
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"##Latest Web Trends\n",
"\n",
"* **Mental Health Awareness**: The web is increasingly becoming a platform for mental health awareness, with many organizations and individuals using social media to share their experiences and promote self-care.\n",
"* **Sustainable Living**: With the growing concern for the environment, sustainable living is becoming more prominent on the web. People are sharing tips and resources on reducing their carbon footprint and living eco-friendly lifestyles.\n",
"* **Artificial Intelligence**: Artificial intelligence is transforming the web, with many companies and developers exploring its applications in areas such as chatbots, virtual assistants, and personalized content recommendations.\n",
"\n",
"##Recent Web News\n",
"\n",
"* A recent study found that 75% of online adults in the US are using social media to manage their mental health.\n",
"* A new platform, called \"Mental Health Matters,\" is launching a series of online courses and resources to promote mental health awareness and support.\n",
"* The web is also seeing an increase in online shopping, with many retailers launching their e-commerce platforms in recent months.\n",
"\n",
"##Search:yes\n"
]
}
],
"execution_count": 35
},
{
"metadata": {},
"cell_type": "code",
"outputs": [],
"execution_count": null,
"source": "",
"id": "4ed1bf189a78e1b3"
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

1778
million-param.ipynb Normal file

File diff suppressed because it is too large Load Diff