Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Sign in / Register
Toggle navigation
B
Basedformer
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Locked Files
Issues
0
Issues
0
List
Boards
Labels
Service Desk
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Security & Compliance
Security & Compliance
Dependency List
License Compliance
Packages
Packages
List
Container Registry
Analytics
Analytics
CI / CD
Code Review
Insights
Issues
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
novelai-storage
Basedformer
Commits
3b246976
Commit
3b246976
authored
Mar 31, 2022
by
novelailab
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
bfloat16 model again
parent
5ff36559
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
2 deletions
+2
-2
hypertrain.py
hypertrain.py
+2
-2
No files found.
hypertrain.py
View file @
3b246976
...
@@ -146,7 +146,7 @@ class HyperNetworkSingle(nn.Module):
...
@@ -146,7 +146,7 @@ class HyperNetworkSingle(nn.Module):
#x = shift_tokens(x, self.num_shifts)
#x = shift_tokens(x, self.num_shifts)
x
=
self
.
linear
(
x
)
x
=
self
.
linear
(
x
)
x
=
x
.
mul
(
torch
.
sigmoid
(
x
))
x
=
x
.
mul
(
torch
.
sigmoid
(
x
))
return
x
.
half
()
return
x
.
bfloat16
()
model_config
=
{
model_config
=
{
...
@@ -194,7 +194,7 @@ gas = train_config["gas"]
...
@@ -194,7 +194,7 @@ gas = train_config["gas"]
Path
(
train_config
[
"save_path"
])
.
mkdir
(
parents
=
True
,
exist_ok
=
True
)
Path
(
train_config
[
"save_path"
])
.
mkdir
(
parents
=
True
,
exist_ok
=
True
)
#model = GPTModel.gpt2_init(model_config).cuda().float()
#model = GPTModel.gpt2_init(model_config).cuda().float()
model
=
load_gpt_j
()
.
cuda
()
.
half
()
model
=
load_gpt_j
()
.
cuda
()
.
bfloat16
()
for
param
in
model
.
parameters
():
for
param
in
model
.
parameters
():
param
.
requires_grad
=
False
param
.
requires_grad
=
False
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment