Skip to content

Commit

Permalink
notes
Browse files Browse the repository at this point in the history
  • Loading branch information
mike dupont committed Mar 23, 2024
1 parent 07f6f84 commit 85a8619
Show file tree
Hide file tree
Showing 19 changed files with 214 additions and 18 deletions.
2 changes: 1 addition & 1 deletion 2023/04/28/gpt4all
Submodule gpt4all updated 44 files
+40 −59 .github/workflows/test-docker-build.yml
+0 −15 docs/source/accelerate/deepspeed.md
+0 −29 docs/source/developer_guides/lora.md
+3 −3 docs/source/developer_guides/quantization.md
+1 −1 docs/source/task_guides/prompt_based_methods.md
+10 −9 examples/causal_language_modeling/peft_lora_clm_accelerate_ds_zero3_offload.py
+1 −1 examples/conditional_generation/peft_adalora_seq2seq.py
+10 −9 examples/conditional_generation/peft_lora_seq2seq_accelerate_ds_zero3_offload.py
+1 −3 examples/feature_extraction/peft_lora_embedding_semantic_search.py
+1 −1 examples/feature_extraction/peft_lora_embedding_semantic_similarity_inference.ipynb
+2 −2 examples/fp4_finetuning/finetune_fp4_opt_bnb_peft.py
+2 −2 examples/int8_training/Finetune_flan_t5_large_bnb_peft.ipynb
+4 −4 examples/int8_training/Finetune_opt_bnb_peft.ipynb
+2 −4 examples/int8_training/fine_tune_blip2_int8.py
+1 −4 examples/int8_training/peft_adalora_whisper_large_training.py
+4 −4 examples/int8_training/peft_bnb_whisper_large_v2_training.ipynb
+2 −2 examples/multi_adapter_examples/PEFT_Multi_LoRA_Inference.ipynb
+1 −1 examples/sft/train.py
+5 −4 examples/sft/utils.py
+1 −1 setup.py
+1 −1 src/peft/__init__.py
+0 −7 src/peft/mixed_model.py
+0 −7 src/peft/peft_model.py
+2 −3 src/peft/tuners/adalora/layer.py
+0 −7 src/peft/tuners/adaption_prompt/config.py
+5 −13 src/peft/tuners/adaption_prompt/layer.py
+2 −12 src/peft/tuners/adaption_prompt/utils.py
+26 −82 src/peft/tuners/lora/bnb.py
+0 −44 src/peft/tuners/lora/config.py
+1 −13 src/peft/tuners/lora/gptq.py
+16 −165 src/peft/tuners/lora/layer.py
+8 −50 src/peft/tuners/lora/model.py
+7 −13 src/peft/tuners/lora/tp_layer.py
+1 −10 src/peft/tuners/prompt_tuning/config.py
+5 −105 src/peft/tuners/tuners_utils.py
+0 −69 src/peft/utils/integrations.py
+1 −1 tests/regression/test_regression.py
+1 −375 tests/test_adaption_prompt.py
+29 −212 tests/test_common_gpu.py
+2 −23 tests/test_custom_models.py
+1 −40 tests/test_decoder_models.py
+18 −294 tests/test_gpu_examples.py
+5 −55 tests/test_initialization.py
+3 −3 tests/test_tuners_utils.py
2 changes: 1 addition & 1 deletion 2023/04/29/gpt4all-ui
2 changes: 1 addition & 1 deletion 2023/07/17/experiments/transformers
Submodule transformers updated 209 files
2 changes: 1 addition & 1 deletion 2024/02/22/nix
Submodule nix updated 191 files
2 changes: 1 addition & 1 deletion 2024/03/12/gnutls
Submodule gnutls updated from a97486 to 7b6856
2 changes: 1 addition & 1 deletion 2024/03/12/guile
Submodule guile updated from 4ec155 to e1690f
2 changes: 1 addition & 1 deletion 2024/03/12/guix-artwork
Submodule guix-artwork updated from d9ea60 to 3101f1
2 changes: 1 addition & 1 deletion 2024/03/13/guile
Submodule guile updated from 6da17c to 4ec155
2 changes: 1 addition & 1 deletion 2024/03/13/guile-gcrypt
Submodule guile-gcrypt updated from d54237 to 25f47d
2 changes: 1 addition & 1 deletion 2024/03/13/guile-json
Submodule guile-json updated 1 files
+11 −0 build.sh
2 changes: 1 addition & 1 deletion 2024/03/13/guile-sqlite3
Submodule guile-sqlite3 updated from 5fa25e to 48cbb2
2 changes: 1 addition & 1 deletion 2024/03/13/guile-zlib
Submodule guile-zlib updated from 13c512 to bc0f48
2 changes: 1 addition & 1 deletion 2024/03/13/libgit2
Submodule libgit2 updated 1 files
+10 −0 build.sh
2 changes: 1 addition & 1 deletion 2024/03/13/scheme-bytestructures
2 changes: 1 addition & 1 deletion 2024/03/18/guix-cuirass
Submodule guix-cuirass updated 1 files
+1,373 −0 aclocal.m4
196 changes: 196 additions & 0 deletions 2024/03/22/notes.org
Original file line number Diff line number Diff line change
@@ -0,0 +1,196 @@
* idea of the day
size as a complex function.
size as a simple function.
size as a simple expression.
size as a simple constant eval expression.
size as a simple constant expression.
size as a simple constant.
size as a constant.
constant size.
equivalent constant size.
constant size that determines non constant size.
constant size that determines the dependent type
dynamically sized objects depending on constant sized inputs.
functionally dependent types.
functionally dependent algebraic types.
equivalent functionally dependent algebraic types.
ooda loop of function construction.
ooda loop of network training.
neural network approximates function.
neural network approximates size function.
neural network training on function.
neural network training on kernel function.
neural network training on compositions of kernel functions.

size of network as capacity measure.
composing networks out of networks.
translating between grammars.
representing 3 languages in terms of each other.
a in terms of b.
b in terms of a.
b in terms of c.
a in terms of b in terms of c. transitive.
different algebras.

* introspection as a process or practice.
mindfullness as a practice.
a practice as a process.
a process as an ooda loop.
introspection as ooda loop.
mindfullness as ooda loop.
the action is either non action or stop.
the effortless action is non action.
the non action is the final resting point of the network that has converged
and needs no updates or minimal updates.
observation of updates to network.
observation of observation of network.
observation of network of observations.
observation as a network.
network observing itself.
mind observing itself.
mind changing itself.
bias towards growth.
growth mindset.
growth function.
bias function.
functional bias.
bias towards functions.
bias towards functional approach.
functions as networks.
functions of functions as networks.
functions of networks.
network functions.
math functions.
function of functions.
function of functions of functions.
game in game.
game in game in game.
simulation
simulation of simulation
simulation of simulation of simulation
co-simulation
multi simulation
simulation in game
creating artifacts in game
composing functions in game
composing functions in simulation
simulation of composing functions in simulation
composing simulations
composing stories
composition
writing your own story.
autopoetic.
limited by words, grammars, functions, biases.
capacity of a given system.
size of a system.
alpha go.
policy network, value network.
size network, type network, function network.
ast network
language network
grammar network
languages network
language networks
grammars network
grammar networks
asts network
ast networks
network network
network networks
networks networks
inter networks of inter networks
universes of universes
simulations of simulations
thoughts about thoughts.
derivative functions.
loss functions
nn training as meta narratives.
build system as meta narratives.
build system as management system
build management system as work management system
functional build management system as functional work management system
gnu meta narrative as one open narrative.
functional narratives.
functional meta narratives.
meta narratives.
narratives. interpretations. ooda. biosemiotics.
going inside.
introspection.
opening labeled boxes.
boxing types.
parameterized types.
abstract types.
type compositions.
composition types.
narrative copositions.
games.
executable functions.
executable decomposable functions.
continuously decomposable functions.
eyeball.
curved sphere.
reflections of self.
reflections of light.
sources of light.
sounds of birds.
thoughts ending in self.
reflections of self.
thoughts about self.
experiences about self.

deep experience of being.
having fun. flow state.
stopping of thoughts.
layers of thoughts and language.
input layer.
network boundries.
experiences about experiences of self.
self discipline.
practice.
process.
matching job applicant search.
job description composition .
functional job description composition.
service matching.
function matching.
description of job to compose system.
compose job to compose system.
compose system to compose job
compose system to compose job description
compose job description to compose job descriptions.
describe job.
job description.

* build server

now for the build server.
First we need to be able to reproduce it's installation.
install the build server.
document the build.
build it again on a new server.

use the build server to build itself.
use the build server to deploy itself.
maybe use guix subst server.

get the shepard service running and building.
be able to inspect the build progress.
be able to insert observational tools into the build.

* ideas from today:

fundamental functions of asts as typeclasses:

1. size (precision)
2. location
3. name
4. type
5. implementation/declaration
6. compose

each of these can be seen a dependent types that connect the result.
the location is for example managed by the compiler, the user and other factors,
available resources.

the size determines or parameterized or is related to the type.
dynamic sizes can be constructed.

0 comments on commit 85a8619

Please sign in to comment.