add2c84a28
The '--reuse|--reuse_maximum' feature mirrors the remote
shared repository and imports all debs from the mirror
to the local repository, there are no deb packages in the
local build directory for the reused packages which make
some tasks like building docker images and secure boot
signing fail for the missing deb packages.
This commit supports the below functions to fix the above
issues:
a. If '--dl_reused' option is enabled for option '--reuse'
or '--reuse_maximum', all the reused deb packages will be
downloaded to their local build directory.
b. 'never_reuse.lst' will be checked and the packages listed
in it will be built locally instead of reusing them if the
option '--reuse' is enabled. And it will be ignored if the
option '--reuse_maximum' is enabled.
Test Plan:
Pass: build-pkgs (make sure the normal build-pkgs works)
Pass: export STX_SHARED_REPO=<url to shard repo>
export STX_SHARED_SOURCE=<url to shared source>
build-pkgs --reuse
Pass: export STX_SHARED_REPO=<url to shard repo>
export STX_SHARED_SOURCE=<url to shared source>
build-pkgs --reuse --dl_reused
Pass: export STX_SHARED_REPO=<url to shard repo>
export STX_SHARED_SOURCE=<url to shared source>
build-pkgs --clean --reuse --dl_reused
Run the secure boot signing script
Pass: export STX_SHARED_REPO=<url to shard repo>
export STX_SHARED_SOURCE=<url to shared source>
build-pkgs --clean --reuse --dl_reused
build-pkgs (Make sure this build will not build from scratch)
Pass: export STX_SHARED_REPO=<url to shard repo>
export STX_SHARED_SOURCE=<url to shared source>
build-pkgs --clean --reuse_maximum --dl_reused
Pass: export STX_SHARED_REPO=<url to shard repo>
export STX_SHARED_SOURCE=<url to shared source>
build-pkgs --reuse_maximum --dl_reused
Partial-Bug: 2017763
Signed-off-by: hqbai <haiqing.bai@windriver.com>
Change-Id: I8cd84dbe6fe8f0262dde12befb0b16367e261968
72 lines
2.5 KiB
Python
Executable File
72 lines
2.5 KiB
Python
Executable File
#!/usr/bin/python3
|
|
|
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
# you may not use this file except in compliance with the License.
|
|
# You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
#
|
|
# Copyright (C) 2021 Wind River Systems,Inc
|
|
import os
|
|
import pickle
|
|
|
|
|
|
def get_pkg_by_deb(clue, debname, logger):
|
|
try:
|
|
with open(clue, 'rb') as fclue:
|
|
try:
|
|
debs = pickle.load(fclue)
|
|
for pkgname, subdebs in debs.items():
|
|
if debname in subdebs:
|
|
return pkgname
|
|
except (EOFError, ValueError, AttributeError, ImportError, IndexError, pickle.UnpicklingError) as e:
|
|
logger.error(str(e))
|
|
logger.warn(f"debs_entry:failed to load {clue}, return None")
|
|
except IOError:
|
|
logger.warn(f"debs_entry:{clue} does not exist")
|
|
return None
|
|
|
|
|
|
def get_subdebs(clue, package, logger):
|
|
try:
|
|
with open(clue, 'rb') as fclue:
|
|
try:
|
|
debs = pickle.load(fclue)
|
|
if package in debs.keys():
|
|
return debs[package]
|
|
except (EOFError, ValueError, AttributeError, ImportError, IndexError, pickle.UnpicklingError) as e:
|
|
logger.warn(f"debs_entry:failed to load {clue}, return None")
|
|
except IOError:
|
|
logger.warn(f"debs_entry:{clue} does not exist")
|
|
return None
|
|
|
|
|
|
def set_subdebs(clue, package, debs, logger):
|
|
debmap = {}
|
|
try:
|
|
with open(clue, 'rb') as fclue:
|
|
try:
|
|
debmap = pickle.load(fclue)
|
|
logger.debug(f"debs_entry:loaded the debs clue {clue}")
|
|
except (EOFError, ValueError, AttributeError, ImportError, IndexError, pickle.UnpicklingError) as e:
|
|
logger.warn(f"debs_entry:failed to load {clue}, recreate it")
|
|
os.remove(clue)
|
|
debmap = {}
|
|
except IOError:
|
|
logger.debug(f"debs_entry:{clue} does not exist")
|
|
|
|
debmap[package] = debs
|
|
try:
|
|
with open(clue, 'wb+') as fclue:
|
|
pickle.dump(debmap, fclue, pickle.HIGHEST_PROTOCOL)
|
|
except IOError:
|
|
raise Exception(f"debs_entry:failed to write {clue}")
|
|
|
|
return True
|