Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Sign in / Register
Toggle navigation
S
Stable Diffusion Webui
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Locked Files
Issues
0
Issues
0
List
Boards
Labels
Service Desk
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Security & Compliance
Security & Compliance
Dependency List
License Compliance
Packages
Packages
List
Container Registry
Analytics
Analytics
CI / CD
Code Review
Insights
Issues
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
novelai-storage
Stable Diffusion Webui
Commits
e334758e
Commit
e334758e
authored
May 11, 2023
by
AUTOMATIC
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
repair #10266
parent
c9e5b921
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
13 deletions
+5
-13
modules/sub_quadratic_attention.py
modules/sub_quadratic_attention.py
+5
-13
No files found.
modules/sub_quadratic_attention.py
View file @
e334758e
...
...
@@ -201,23 +201,15 @@ def efficient_dot_product_attention(
key
=
key
,
value
=
value
,
)
# slices of res tensor are mutable, modifications made
# to the slices will affect the original tensor.
# if output of compute_query_chunk_attn function has same number of
# dimensions as input query tensor, we initialize tensor like this:
num_query_chunks
=
int
(
np
.
ceil
(
q_tokens
/
query_chunk_size
))
query_shape
=
get_query_chunk
(
0
)
.
shape
res_shape
=
(
query_shape
[
0
],
query_shape
[
1
]
*
num_query_chunks
,
*
query_shape
[
2
:])
res_dtype
=
get_query_chunk
(
0
)
.
dtype
res
=
torch
.
zeros
(
res_shape
,
dtype
=
res_dtype
)
for
i
in
range
(
num_query_chunks
):
res
=
torch
.
zeros_like
(
query
)
for
i
in
range
(
math
.
ceil
(
q_tokens
/
query_chunk_size
)):
attn_scores
=
compute_query_chunk_attn
(
query
=
get_query_chunk
(
i
*
query_chunk_size
),
key
=
key
,
value
=
value
,
)
res
[:,
i
*
query_chunk_size
:(
i
+
1
)
*
query_chunk_size
,
:]
=
attn_scores
res
[:,
i
*
query_chunk_size
:
i
*
query_chunk_size
+
attn_scores
.
shape
[
1
],
:]
=
attn_scores
return
res
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment