Compare commits

...

31 Commits

Author SHA1 Message Date
WJQSERVER
d2b2d823b8 3.4.0
Add support for multi-target docker image(oci) proxy (3.4.0)
Add Hub theme
2025-05-21 16:24:08 +08:00
wjqserver
4598257faa update readme.md 2025-05-21 16:07:07 +08:00
wjqserver
1afb352194 3.4.0 2025-05-21 15:24:05 +08:00
wjqserver
430e313d47 avoid nil *ptr & fix path 2025-05-21 12:08:17 +08:00
wjqserver
31d435bfa0 add oci proxy & nest shell api 2025-05-21 11:55:04 +08:00
wjqserver
6ff23f639e add hub theme & add more check for wcache close 2025-05-21 11:54:43 +08:00
wjqserver
c7954ae91a 25w40a 2025-05-21 09:03:14 +08:00
wjqserver
11099176bf add support for multi-target docker image(oci) proxy 2025-05-21 09:03:00 +08:00
WJQSERVER
f3eb92ea51 Merge pull request #107 from WJQSERVER-STUDIO/dev
3.3.3
2025-05-20 10:10:19 +08:00
wjqserver
5ddbf1d2a0 3.3.3 2025-05-20 10:05:55 +08:00
wjqserver
d38ca3969f revert route handle for 3.3.x 2025-05-20 10:03:48 +08:00
wjqserver
146b0d7748 update nest 2025-05-19 12:16:20 +08:00
wjqserver
d92424cb94 25w39a 2025-05-19 12:00:36 +08:00
WJQSERVER
0f437dc891 Merge pull request #106 from WJQSERVER-STUDIO/dev
3.3.2
2025-05-18 07:05:19 +08:00
wjqserver
816b35654a update readme.md 2025-05-18 06:20:55 +08:00
wjqserver
a4fae95526 3.3.2 2025-05-18 06:13:00 +08:00
wjqserver
ea0e4e9801 change the default theme to design 2025-05-18 06:11:44 +08:00
wjqserver
5facc36947 update docs 2025-05-18 06:09:04 +08:00
WJQSERVER
5c25bc012f Merge pull request #105 from WJQSERVER-STUDIO/dev
3.3.1
2025-05-16 19:54:20 +08:00
wjqserver
b2712f8184 3.3.1 2025-05-16 19:53:48 +08:00
wjqserver
566a0ea26a 25w37a 2025-05-16 19:28:08 +08:00
wjqserver
7d4aae1668 merge customTarget into target 2025-05-16 00:24:57 +08:00
wjqserver
052243b095 add customTarget 2025-05-16 00:15:04 +08:00
wjqserver
4ded2186d8 update deps 2025-05-15 18:50:36 +08:00
WJQSERVER
aa95daf8c0 Merge pull request #103 from WJQSERVER-STUDIO/dev
3.3.0
2025-05-15 18:46:29 +08:00
wjqserver
89b850c1ec 3.3.0 2025-05-15 18:45:52 +08:00
wjqserver
ce814875e1 25w36d 2025-05-14 17:55:37 +08:00
wjqserver
47c03763a7 25w36c 2025-05-14 01:34:05 +08:00
wjqserver
71bc2aaed7 add bandwidth limiter 2025-05-14 01:33:54 +08:00
wjqserver
3f8d16511e 25w36b 2025-05-13 19:04:21 +08:00
wjqserver
43469532d4 25w36a 2025-05-13 14:51:34 +08:00
22 changed files with 1232 additions and 280 deletions

4
.gitignore vendored
View File

@@ -4,4 +4,6 @@ demo.toml
*.bak
list.json
repos
pages
pages
*_test
.*

View File

@@ -1,5 +1,80 @@
# 更新日志
3.4.0 - 2025-05-21
---
- ADD: 初步实现多`target` Docker代理
- ADD: 加入`weakcache`用于处理短期令牌
- ADD: 新增`hub`主题
- ADD: 新增`/api/shell_nest/status``/api/oci_proxy/status` API
25w40b - 2025-05-21
---
- PRE-RELEASE: 此版本是v3.4.0预发布版本,请勿在生产环境中使用;
- ADD: 新增`hub`主题
- ADD: 新增`/api/shell_nest/status``/api/oci_proxy/status` API
- CHANGE: 对细节进行优化
25w40a - 2025-05-21
---
- PRE-RELEASE: 此版本是v3.4.0预发布版本,请勿在生产环境中使用;
- ADD: 初步实现多`target` Docker代理
- ADD: 加入`weakcache`用于处理短期令牌
3.3.3 - 2025-05-20
---
- CHANGE: 加入`senseClientDisconnection``async`配置项
25w39a - 2025-05-19
---
- PRE-RELEASE: 此版本是v3.3.3预发布版本,请勿在生产环境中使用;
- CHANGE: 加入`senseClientDisconnection``async`配置项
3.3.2 - 2025-05-18
---
- CHANGE: 默认主题改为`design`
25w38a - 2025-05-18
---
- PRE-RELEASE: 此版本是v3.3.2预发布版本,请勿在生产环境中使用;
- CHANGE: 默认主题改为`design`
3.3.1 - 2025-05-16
- CHANGE: 为`target`放宽限制, 支持自定义
- CHANGE: 更新`hertz`, `0.9.7`=>`0.10.0`
25w37a - 2025-05-16
---
- PRE-RELEASE: 此版本是v3.3.1预发布版本,请勿在生产环境中使用;
- CHANGE: 为`target`放宽限制, 支持自定义
- CHANGE: 更新`hertz`, `0.9.7`=>`0.10.0`
3.3.0 - 2025-05-15
---
- CHANGE: 为`httpc`加入`request builder``withcontext`选项
- ADD: 加入带宽限制功能
- ADD: 为`netpoll`模式开启探测客户端是否断开功能
25w36d - 2025-05-14
---
- PRE-RELEASE: 此版本是v3.3.0预发布版本,请勿在生产环境中使用;
- ADD: 为`netpoll`模式开启探测客户端是否断开功能
25w36c - 2025-05-14
---
- PRE-RELEASE: 此版本是v3.3.0预发布版本,请勿在生产环境中使用;
- ADD: 加入带宽限制功能
- CHANGE: 将`httpc`切换回主分支, `25w36b`测试的部分已被合入`httpc`主线
25w36b - 2025-05-13
---
- PRE-RELEASE: 此版本是v3.3.0预发布版本,请勿在生产环境中使用;
- CHANGE: `httpc`切换到`dev`, 测试在retry前检查ctx状态
25w36a - 2025-05-13
---
- PRE-RELEASE: 此版本是v3.3.0预发布版本,请勿在生产环境中使用;
- CHANGE: 为`httpc`加入`request builder``withcontext`选项
3.2.4 - 2025-05-13
---
- CHANGE: 移除未使用的变量与相关计算

View File

@@ -1 +1 @@
25w35a
25w40b

View File

@@ -17,12 +17,13 @@
- 🌐 **使用字节旗下的 [HertZ](https://github.com/cloudwego/hertz) 作为 Web 框架**
- 📡 **使用 [Touka-HTTPC](https://github.com/satomitouka/touka-httpc) 作为 HTTP 客户端**
- 📥 **支持 Git clone、raw、releases 等文件拉取**
- 🐳 **支持反代Docker, GHCR等镜像仓库**
- 🎨 **支持多个前端主题**
- 🚫 **支持自定义黑名单/白名单**
- 🗄️ **支持 Git Clone 缓存(配合 [Smart-Git](https://github.com/WJQSERVER-STUDIO/smart-git)**
- 🐳 **支持 Docker 部署**
- 🐳 **支持自托管**
- 🐳 **支持自托管与Docker容器化部署**
-**支持速率限制**
-**支持带宽速率限制**
- 🔒 **支持用户鉴权**
- 🐚 **支持 shell 脚本多层嵌套加速**
@@ -34,11 +35,13 @@
[相关文章](https://blog.wjqserver.com/categories/my-program/)
[项目文档](https://github.com/WJQSERVER-STUDIO/ghproxy/blob/main/docs/menu.md)
[GHProxy项目文档](https://github.com/WJQSERVER-STUDIO/ghproxy/blob/main/docs/menu.md)
[GHProxy项目文档Next(仍在建设中)](https://wjqserver.pages.dev/docs/category/ghproxy) 感谢 [@redbunnys](https://github.com/redbunnys)的维护
### 使用示例
```
```bash
# 下载文件
https://ghproxy.1888866.xyz/raw.githubusercontent.com/WJQSERVER-STUDIO/tools-stable/main/tools-stable-ghproxy.sh
https://ghproxy.1888866.xyz/https://raw.githubusercontent.com/WJQSERVER-STUDIO/tools-stable/main/tools-stable-ghproxy.sh
@@ -46,6 +49,15 @@ https://ghproxy.1888866.xyz/https://raw.githubusercontent.com/WJQSERVER-STUDIO/t
# 克隆仓库
git clone https://ghproxy.1888866.xyz/github.com/WJQSERVER-STUDIO/ghproxy.git
git clone https://ghproxy.1888866.xyz/https://github.com/WJQSERVER-STUDIO/ghproxy.git
# Docker(OCI) 代理
docker pull gh.example.com/wjqserver/ghproxy
docker pull gh.example.com/adguard/adguardhome
docker pull gh.example.com/docker.io/wjqserver/ghproxy
docker pull gh.example.com/docker.io/adguard/adguardhome
docker pull gh.example.com/ghcr.io/openfaas/queue-worker
```
## 部署说明

View File

@@ -1 +1 @@
3.2.4
3.4.0

View File

@@ -49,14 +49,18 @@ func InitHandleRouter(cfg *config.Config, r *server.Hertz, version string) {
apiRouter.GET("/smartgit/status", func(ctx context.Context, c *app.RequestContext) {
SmartGitStatusHandler(cfg, c, ctx)
})
apiRouter.GET("/shell_nest/status", func(ctx context.Context, c *app.RequestContext) {
shellNestStatusHandler(cfg, c, ctx)
})
apiRouter.GET("/oci_proxy/status", func(ctx context.Context, c *app.RequestContext) {
ociProxyStatusHandler(cfg, c, ctx)
})
}
logInfo("API router Init success")
}
func SizeLimitHandler(cfg *config.Config, c *app.RequestContext, ctx context.Context) {
sizeLimit := cfg.Server.SizeLimit
logInfo("%s %s %s %s %s", c.ClientIP(), c.Method(), string(c.Path()), c.Request.Header.UserAgent(), c.Request.Header.GetProtocol())
c.Response.Header.Set("Content-Type", "application/json")
c.JSON(200, (map[string]interface{}{
"MaxResponseBodySize": sizeLimit,
@@ -64,7 +68,6 @@ func SizeLimitHandler(cfg *config.Config, c *app.RequestContext, ctx context.Con
}
func WhiteListStatusHandler(cfg *config.Config, c *app.RequestContext, ctx context.Context) {
logInfo("%s %s %s %s %s", c.ClientIP(), c.Method(), string(c.Path()), c.Request.Header.UserAgent(), c.Request.Header.GetProtocol())
c.Response.Header.Set("Content-Type", "application/json")
c.JSON(200, (map[string]interface{}{
"Whitelist": cfg.Whitelist.Enabled,
@@ -72,7 +75,6 @@ func WhiteListStatusHandler(cfg *config.Config, c *app.RequestContext, ctx conte
}
func BlackListStatusHandler(cfg *config.Config, c *app.RequestContext, ctx context.Context) {
logInfo("%s %s %s %s %s", c.ClientIP(), c.Method(), string(c.Path()), c.Request.Header.UserAgent(), c.Request.Header.GetProtocol())
c.Response.Header.Set("Content-Type", "application/json")
c.JSON(200, (map[string]interface{}{
"Blacklist": cfg.Blacklist.Enabled,
@@ -80,7 +82,6 @@ func BlackListStatusHandler(cfg *config.Config, c *app.RequestContext, ctx conte
}
func CorsStatusHandler(cfg *config.Config, c *app.RequestContext, ctx context.Context) {
logInfo("%s %s %s %s %s", c.ClientIP(), c.Method(), string(c.Path()), c.Request.Header.UserAgent(), c.Request.Header.GetProtocol())
c.Response.Header.Set("Content-Type", "application/json")
c.JSON(200, (map[string]interface{}{
"Cors": cfg.Server.Cors,
@@ -88,7 +89,6 @@ func CorsStatusHandler(cfg *config.Config, c *app.RequestContext, ctx context.Co
}
func HealthcheckHandler(c *app.RequestContext, ctx context.Context) {
logInfo("%s %s %s %s %s", c.ClientIP(), c.Method(), string(c.Path()), c.Request.Header.UserAgent(), c.Request.Header.GetProtocol())
c.Response.Header.Set("Content-Type", "application/json")
c.JSON(200, (map[string]interface{}{
"Status": "OK",
@@ -96,7 +96,6 @@ func HealthcheckHandler(c *app.RequestContext, ctx context.Context) {
}
func VersionHandler(c *app.RequestContext, ctx context.Context, version string) {
logInfo("%s %s %s %s %s", c.ClientIP(), c.Method(), string(c.Path()), c.Request.Header.UserAgent(), c.Request.Header.GetProtocol())
c.Response.Header.Set("Content-Type", "application/json")
c.JSON(200, (map[string]interface{}{
"Version": version,
@@ -104,7 +103,6 @@ func VersionHandler(c *app.RequestContext, ctx context.Context, version string)
}
func RateLimitStatusHandler(cfg *config.Config, c *app.RequestContext, ctx context.Context) {
logInfo("%s %s %s %s %s", c.ClientIP(), c.Method(), string(c.Path()), c.Request.Header.UserAgent(), c.Request.Header.GetProtocol())
c.Response.Header.Set("Content-Type", "application/json")
c.JSON(200, (map[string]interface{}{
"RateLimit": cfg.RateLimit.Enabled,
@@ -112,7 +110,6 @@ func RateLimitStatusHandler(cfg *config.Config, c *app.RequestContext, ctx conte
}
func RateLimitLimitHandler(cfg *config.Config, c *app.RequestContext, ctx context.Context) {
logInfo("%s %s %s %s %s", c.ClientIP(), c.Method(), string(c.Path()), c.Request.Header.UserAgent(), c.Request.Header.GetProtocol())
c.Response.Header.Set("Content-Type", "application/json")
c.JSON(200, (map[string]interface{}{
"RatePerMinute": cfg.RateLimit.RatePerMinute,
@@ -120,9 +117,23 @@ func RateLimitLimitHandler(cfg *config.Config, c *app.RequestContext, ctx contex
}
func SmartGitStatusHandler(cfg *config.Config, c *app.RequestContext, ctx context.Context) {
logInfo("%s %s %s %s %s", c.ClientIP(), c.Method(), string(c.Path()), c.Request.Header.UserAgent(), c.Request.Header.GetProtocol())
c.Response.Header.Set("Content-Type", "application/json")
c.JSON(200, (map[string]interface{}{
"enabled": cfg.GitClone.Mode == "cache",
}))
}
func shellNestStatusHandler(cfg *config.Config, c *app.RequestContext, ctx context.Context) {
c.Response.Header.Set("Content-Type", "application/json")
c.JSON(200, (map[string]interface{}{
"enabled": cfg.Shell.Editor,
}))
}
func ociProxyStatusHandler(cfg *config.Config, c *app.RequestContext, ctx context.Context) {
c.Response.Header.Set("Content-Type", "application/json")
c.JSON(200, (map[string]interface{}{
"enabled": cfg.Docker.Enabled,
"target": cfg.Docker.Target,
}))
}

View File

@@ -34,14 +34,15 @@ debug = false
*/
type ServerConfig struct {
Port int `toml:"port"`
Host string `toml:"host"`
NetLib string `toml:"netlib"`
SizeLimit int `toml:"sizeLimit"`
MemLimit int64 `toml:"memLimit"`
H2C bool `toml:"H2C"`
Cors string `toml:"cors"`
Debug bool `toml:"debug"`
Port int `toml:"port"`
Host string `toml:"host"`
NetLib string `toml:"netlib"`
SenseClientDisconnection bool `toml:"senseClientDisconnection"`
SizeLimit int `toml:"sizeLimit"`
MemLimit int64 `toml:"memLimit"`
H2C bool `toml:"H2C"`
Cors string `toml:"cors"`
Debug bool `toml:"debug"`
}
/*
@@ -98,6 +99,7 @@ type LogConfig struct {
LogFilePath string `toml:"logFilePath"`
MaxLogSize int `toml:"maxLogSize"`
Level string `toml:"level"`
Async bool `toml:"async"`
HertZLogPath string `toml:"hertzLogPath"`
}
@@ -129,11 +131,35 @@ type WhitelistConfig struct {
WhitelistFile string `toml:"whitelistFile"`
}
/*
[rateLimit]
enabled = false
rateMethod = "total" # "total" or "ip"
ratePerMinute = 100
burst = 10
[rateLimit.bandwidthLimit]
enabled = false
totalLimit = "100mbps"
totalBurst = "100mbps"
singleLimit = "10mbps"
singleBurst = "10mbps"
*/
type RateLimitConfig struct {
Enabled bool `toml:"enabled"`
RateMethod string `toml:"rateMethod"`
RatePerMinute int `toml:"ratePerMinute"`
Burst int `toml:"burst"`
Enabled bool `toml:"enabled"`
RateMethod string `toml:"rateMethod"`
RatePerMinute int `toml:"ratePerMinute"`
Burst int `toml:"burst"`
BandwidthLimit BandwidthLimitConfig
}
type BandwidthLimitConfig struct {
Enabled bool `toml:"enabled"`
TotalLimit string `toml:"totalLimit"`
TotalBurst string `toml:"totalBurst"`
SingleLimit string `toml:"singleLimit"`
SingleBurst string `toml:"singleBurst"`
}
/*
@@ -252,6 +278,13 @@ func DefaultConfig() *Config {
RateMethod: "total",
RatePerMinute: 100,
Burst: 10,
BandwidthLimit: BandwidthLimitConfig{
Enabled: false,
TotalLimit: "100mbps",
TotalBurst: "100mbps",
SingleLimit: "10mbps",
SingleBurst: "10mbps",
},
},
Outbound: OutboundConfig{
Enabled: false,

View File

@@ -2,6 +2,7 @@
host = "0.0.0.0"
port = 8080
netlib = "netpoll" # "netpoll" / "std" "standard" "net/http" "net"
senseClientDisconnection = false
sizeLimit = 125 # MB
memLimit = 0 # MB
H2C = true
@@ -33,6 +34,7 @@ staticDir = "/data/www"
logFilePath = "/data/ghproxy/log/ghproxy.log"
maxLogSize = 5 # MB
level = "info" # dump, debug, info, warn, error, none
async = false
hertzLogPath = "/data/ghproxy/log/hertz.log"
[auth]
@@ -57,6 +59,13 @@ rateMethod = "total" # "ip" or "total"
ratePerMinute = 180
burst = 5
[rateLimit.bandwidthLimit]
enabled = false
totalLimit = "100mbps"
totalBurst = "100mbps"
singleLimit = "10mbps"
singleBurst = "10mbps"
[outbound]
enabled = false
url = "socks5://127.0.0.1:1080" # "http://127.0.0.1:7890"

View File

@@ -68,13 +68,20 @@ rateMethod = "total" # "ip" or "total"
ratePerMinute = 180
burst = 5
[rateLimit.bandwidthLimit]
enabled = false
totalLimit = "100mbps"
totalBurst = "100mbps"
singleLimit = "10mbps"
singleBurst = "10mbps"
[outbound]
enabled = false
url = "socks5://127.0.0.1:1080" # "http://127.0.0.1:7890"
[docker]
enabled = false
target = "ghcr" # ghcr/dockerhub
target = "ghcr" # ghcr/dockerhub or "xx.example.com"
```
### 配置项详细说明
@@ -291,6 +298,27 @@ target = "ghcr" # ghcr/dockerhub
* 类型: 整数 (`int`)
* 默认值: `5`
* 说明: 允许在短时间内超过 `ratePerMinute` 的突发请求数。
* **`[rateLimit.bandwidthLimit]` 带宽速率限制**
* `enabled`: 是否启用带宽速率限制。
* 类型: 布尔值 (`bool`)
* 默认值: `false` (禁用)
* 说明: 启用后,`ghproxy` 将根据配置的策略限制带宽使用,防止服务被滥用。
* `totalLimit`: 全局带宽限制。
* 类型: 字符串 (`string`)
* 默认值: `"100mbps"`
* 说明: 设置全局最大带宽使用量。支持的单位有 "kbps", "mbps", "gbps"。
* `totalBurst`: 全局突发带宽。
* 类型: 字符串 (`string`)
* 默认值: `"100mbps"`
* 说明: 设置全局突发带宽使用量。支持的单位有 "kbps", "mbps", "gbps"。
* `singleLimit`: 单个连接带宽限制。
* 类型: 字符串 (`string`)
* 默认值: `"10mbps"`
* 说明: 设置单个连接的最大带宽使用量。支持的单位有 "kbps", "mbps", "gbps"。
* `singleBurst`: 单个连接突发带宽。
* 类型: 字符串 (`string`)
* 默认值: `"10mbps"`
* 说明: 设置单个连接的突发带宽使用量。支持的单位有 "kbps", "mbps", "gbps"。
* **`[outbound]` - 出站代理配置**
@@ -318,6 +346,7 @@ target = "ghcr" # ghcr/dockerhub
* 说明: 指定要代理的 Docker 注册表。
* `"ghcr"`: 代理 GitHub Container Registry (ghcr.io)。
* `"dockerhub"`: 代理 Docker Hub (docker.io)。
* 自定义, 支持传入自定义target, 例如`"docker.example.com"`
## `blacklist.json` - 黑名单配置

13
go.mod
View File

@@ -4,17 +4,19 @@ go 1.24.3
require (
github.com/BurntSushi/toml v1.5.0
github.com/WJQSERVER-STUDIO/httpc v0.5.0
github.com/WJQSERVER-STUDIO/logger v1.6.0
github.com/cloudwego/hertz v0.9.7
github.com/WJQSERVER-STUDIO/httpc v0.5.1
github.com/WJQSERVER-STUDIO/logger v1.7.1
github.com/cloudwego/hertz v0.10.0
github.com/hertz-contrib/http2 v0.1.8
golang.org/x/net v0.40.0
golang.org/x/time v0.11.0
)
require github.com/WJQSERVER-STUDIO/go-utils/limitreader v0.0.2
require (
github.com/WJQSERVER-STUDIO/go-utils/copyb v0.0.4 // indirect
github.com/WJQSERVER-STUDIO/go-utils/log v0.0.2 // indirect
github.com/WJQSERVER-STUDIO/go-utils/log v0.0.3 // indirect
github.com/bytedance/gopkg v0.1.2 // indirect
github.com/bytedance/sonic v1.13.2 // indirect
github.com/bytedance/sonic/loader v0.2.4 // indirect
@@ -36,3 +38,6 @@ require (
golang.org/x/text v0.25.0 // indirect
google.golang.org/protobuf v1.36.6 // indirect
)
//replace github.com/WJQSERVER-STUDIO/httpc v0.5.1 => /data/github/WJQSERVER-STUDIO/httpc
//replace github.com/WJQSERVER-STUDIO/logger v1.6.0 => /data/github/WJQSERVER-STUDIO/logger

18
go.sum
View File

@@ -2,12 +2,14 @@ github.com/BurntSushi/toml v1.5.0 h1:W5quZX/G/csjUnuI8SUYlsHs9M38FC7znL0lIO+DvMg
github.com/BurntSushi/toml v1.5.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/WJQSERVER-STUDIO/go-utils/copyb v0.0.4 h1:JLtFd00AdFg/TP+dtvIzLkdHwKUGPOAijN1sMtEYoFg=
github.com/WJQSERVER-STUDIO/go-utils/copyb v0.0.4/go.mod h1:FZ6XE+4TKy4MOfX1xWKe6Rwsg0ucYFCdNh1KLvyKTfc=
github.com/WJQSERVER-STUDIO/go-utils/log v0.0.2 h1:9CSf+V0ZQPl2ijC/g6v/ObemmhpKcikKVIodsaLExTA=
github.com/WJQSERVER-STUDIO/go-utils/log v0.0.2/go.mod h1:j9Q+xnwpOfve7/uJnZ2izRQw6NNoXjvJHz7vUQAaLZE=
github.com/WJQSERVER-STUDIO/httpc v0.5.0 h1:0yJA+dOgbnO3R/mAWPjlbUq5lIqaxRV38XfiX3jt6pg=
github.com/WJQSERVER-STUDIO/httpc v0.5.0/go.mod h1:M7KNUZjjhCkzzcg9lBPs9YfkImI+7vqjAyjdA19+joE=
github.com/WJQSERVER-STUDIO/logger v1.6.0 h1:xK2xV7hlkMXaWzvj4+cNoNWA+JfnJaHX6VU+RrPnr7Q=
github.com/WJQSERVER-STUDIO/logger v1.6.0/go.mod h1:TICMsR7geROHBg6rxwkqUNGydo34XVsX93yeoxyfuyY=
github.com/WJQSERVER-STUDIO/go-utils/limitreader v0.0.2 h1:8bBkKk6E2Zr+I5szL7gyc5f0DK8N9agIJCpM1Cqw2NE=
github.com/WJQSERVER-STUDIO/go-utils/limitreader v0.0.2/go.mod h1:yPX8xuZH+py7eLJwOYj3VVI/4/Yuy5+x8Mhq8qezcPg=
github.com/WJQSERVER-STUDIO/go-utils/log v0.0.3 h1:t6nyLhmo9pSfVHm1Wu1WyLsTpXFSjSpQtVKqEDpiZ5Q=
github.com/WJQSERVER-STUDIO/go-utils/log v0.0.3/go.mod h1:j9Q+xnwpOfve7/uJnZ2izRQw6NNoXjvJHz7vUQAaLZE=
github.com/WJQSERVER-STUDIO/httpc v0.5.1 h1:+TKCPYBuj7PAHuiduGCGAqsHAa4QtsUfoVwRN777q64=
github.com/WJQSERVER-STUDIO/httpc v0.5.1/go.mod h1:M7KNUZjjhCkzzcg9lBPs9YfkImI+7vqjAyjdA19+joE=
github.com/WJQSERVER-STUDIO/logger v1.7.1 h1:sAFsF3umimY0Vmue5WnGf1Qxvm/vlhK2srZakWVtlFU=
github.com/WJQSERVER-STUDIO/logger v1.7.1/go.mod h1:cvP0XdFIMLtDWOZeKhklshzipkVU1zufsU4rKNfoM24=
github.com/bytedance/gopkg v0.1.1/go.mod h1:576VvJ+eJgyCzdjS+c4+77QF3p7ubbtiKARP3TxducM=
github.com/bytedance/gopkg v0.1.2 h1:8o2feYuxknDpN+O7kPwvSXfMEKfYvJYiA2K7aonoMEQ=
github.com/bytedance/gopkg v0.1.2/go.mod h1:576VvJ+eJgyCzdjS+c4+77QF3p7ubbtiKARP3TxducM=
@@ -22,8 +24,8 @@ github.com/cloudwego/base64x v0.1.5 h1:XPciSp1xaq2VCSt6lF0phncD4koWyULpl5bUxbfCy
github.com/cloudwego/base64x v0.1.5/go.mod h1:0zlkT4Wn5C6NdauXdJRhSKRlJvmclQ1hhJgA0rcu/8w=
github.com/cloudwego/gopkg v0.1.4 h1:EoQiCG4sTonTPHxOGE0VlQs+sQR+Hsi2uN0qqwu8O50=
github.com/cloudwego/gopkg v0.1.4/go.mod h1:FQuXsRWRsSqJLsMVd5SYzp8/Z1y5gXKnVvRrWUOsCMI=
github.com/cloudwego/hertz v0.9.7 h1:tAVaiO+vTf+ZkQhvNhKbDJ0hmC4oJ7bzwDi1KhvhHy4=
github.com/cloudwego/hertz v0.9.7/go.mod h1:t6d7NcoQxPmETvzPMMIVPHMn5C5QzpqIiFsaavoLJYQ=
github.com/cloudwego/hertz v0.10.0 h1:V0vmBaLdQPlgL6w2TA6PZL1g6SGgQznFx6vqxWdCcKw=
github.com/cloudwego/hertz v0.10.0/go.mod h1:lRBohmcDkGx5TLK6QKFGdzJ6n3IXqGueHsOiXcYgXA4=
github.com/cloudwego/iasm v0.2.0/go.mod h1:8rXZaNYT2n95jn+zTI1sDr+IgcD2GVs0nlbbQPiEFhY=
github.com/cloudwego/netpoll v0.7.0 h1:bDrxQaNfijRI1zyGgXHQoE/nYegL0nr+ijO1Norelc4=
github.com/cloudwego/netpoll v0.7.0/go.mod h1:PI+YrmyS7cIr0+SD4seJz3Eo3ckkXdu2ZVKBLhURLNU=

58
main.go
View File

@@ -17,6 +17,7 @@ import (
"ghproxy/middleware/loggin"
"ghproxy/proxy"
"ghproxy/rate"
"ghproxy/weakcache"
"github.com/WJQSERVER-STUDIO/logger"
"github.com/hertz-contrib/http2/factory"
@@ -50,6 +51,10 @@ var (
pagesFS embed.FS
)
var (
wcache *weakcache.Cache[string] // docker token缓存
)
var (
logw = logger.Logw
logDump = logger.LogDump
@@ -121,6 +126,7 @@ func loadConfig() {
func setupLogger(cfg *config.Config) {
var err error
err = logger.Init(cfg.Log.LogFilePath, cfg.Log.MaxLogSize)
if err != nil {
fmt.Printf("Failed to initialize logger: %v\n", err)
@@ -131,6 +137,8 @@ func setupLogger(cfg *config.Config) {
fmt.Printf("Logger Level Error: %v\n", err)
os.Exit(1)
}
logger.SetAsync(cfg.Log.Async)
fmt.Printf("Log Level: %s\n", cfg.Log.Level)
logDebug("Config File Path: ", cfgfile)
logDebug("Loaded config: %v\n", cfg)
@@ -181,7 +189,11 @@ func setupRateLimit(cfg *config.Config) {
}
func InitReq(cfg *config.Config) {
proxy.InitReq(cfg)
err := proxy.InitReq(cfg)
if err != nil {
fmt.Printf("Failed to initialize request: %v\n", err)
os.Exit(1)
}
}
// loadEmbeddedPages 加载嵌入式页面资源
@@ -201,9 +213,11 @@ func loadEmbeddedPages(cfg *config.Config) (fs.FS, fs.FS, error) {
pages, err = fs.Sub(pagesFS, "pages/classic")
case "mino":
pages, err = fs.Sub(pagesFS, "pages/mino")
case "hub":
pages, err = fs.Sub(pagesFS, "pages/hub")
default:
pages, err = fs.Sub(pagesFS, "pages/bootstrap") // 默认主题
logWarning("Invalid Pages Theme: %s, using default theme 'bootstrap'", cfg.Pages.Theme)
pages, err = fs.Sub(pagesFS, "pages/design") // 默认主题
logWarning("Invalid Pages Theme: %s, using default theme 'design'", cfg.Pages.Theme)
}
if err != nil {
@@ -282,7 +296,7 @@ func setInternalRoute(cfg *config.Config, r *server.Hertz) error {
staticServer.ServeHTTP(adaptor.GetCompatResponseWriter(&c.Response), req)
})
r.GET("/favicon.ico", func(ctx context.Context, c *app.RequestContext) {
staticServer := http.FileServer(http.FS(pages))
staticServer := http.FileServer(http.FS(assets))
req, err := adaptor.GetCompatRequest(&c.Request)
if err != nil {
logError("%s", err)
@@ -353,6 +367,9 @@ func init() {
setMemLimit(cfg)
loadlist(cfg)
setupRateLimit(cfg)
if cfg.Docker.Enabled {
wcache = proxy.InitWeakCache()
}
if cfg.Server.Debug {
runMode = "dev"
@@ -397,11 +414,13 @@ func main() {
r = server.New(
server.WithH2C(true),
server.WithHostPorts(addr),
server.WithSenseClientDisconnection(cfg.Server.SenseClientDisconnection),
)
r.AddProtocol("h2", factory.NewServerFactory())
} else {
r = server.New(
server.WithHostPorts(addr),
server.WithSenseClientDisconnection(cfg.Server.SenseClientDisconnection),
)
}
} else {
@@ -459,10 +478,27 @@ func main() {
proxy.RoutingHandler(cfg, limiter, iplimiter)(ctx, c)
})
r.Any("/v2/*filepath", func(ctx context.Context, c *app.RequestContext) {
proxy.GhcrRouting(cfg)(ctx, c)
r.GET("/v2/", func(ctx context.Context, c *app.RequestContext) {
emptyJSON := "{}"
c.Header("Content-Type", "application/json")
c.Header("Content-Length", fmt.Sprint(len(emptyJSON)))
c.Header("Docker-Distribution-API-Version", "registry/2.0")
c.Status(200)
c.Write([]byte(emptyJSON))
})
r.Any("/v2/:target/:user/:repo/*filepath", func(ctx context.Context, c *app.RequestContext) {
proxy.GhcrWithImageRouting(cfg)(ctx, c)
})
/*
r.Any("/v2/:target/*filepath", func(ctx context.Context, c *app.RequestContext) {
proxy.GhcrRouting(cfg)(ctx, c)
})
*/
r.NoRoute(func(ctx context.Context, c *app.RequestContext) {
proxy.NoRouteHandler(cfg, limiter, iplimiter)(ctx, c)
})
@@ -476,17 +512,21 @@ func main() {
http.ListenAndServe("localhost:6060", nil)
}()
}
if wcache != nil {
defer wcache.StopCleanup()
}
r.Spin()
defer logger.Close()
defer func() {
if hertZfile != nil {
var err error
err = hertZfile.Close()
err := hertZfile.Close()
if err != nil {
logError("Failed to close hertz log file: %v", err)
}
}
}()
r.Spin()
fmt.Println("Program Exit")
}

62
proxy/authparse.go Normal file
View File

@@ -0,0 +1,62 @@
package proxy
import (
"fmt"
"strings"
)
// BearerAuthParams 用于存放解析出的 Bearer 认证参数
type BearerAuthParams struct {
Realm string
Service string
Scope string
}
// parseBearerWWWAuthenticateHeader 解析 Bearer 方案的 Www-Authenticate Header。
// 它期望格式为 'Bearer key1="value1",key2="value2",...'
// 并尝试将已知参数解析到 BearerAuthParams struct 中。
func parseBearerWWWAuthenticateHeader(headerValue string) (*BearerAuthParams, error) {
if headerValue == "" {
return nil, fmt.Errorf("header value is empty")
}
// 检查 Scheme 是否是 "Bearer"
parts := strings.SplitN(headerValue, " ", 2)
if len(parts) != 2 || strings.ToLower(parts[0]) != "bearer" {
return nil, fmt.Errorf("invalid or non-bearer header format: got '%s'", headerValue)
}
paramsStr := parts[1]
paramPairs := strings.Split(paramsStr, ",")
tempMap := make(map[string]string)
for _, pair := range paramPairs {
trimmedPair := strings.TrimSpace(pair)
keyValue := strings.SplitN(trimmedPair, "=", 2)
if len(keyValue) != 2 {
logWarning("Skipping malformed parameter '%s' in Www-Authenticate header: %s", pair, headerValue)
continue
}
key := strings.TrimSpace(keyValue[0])
value := strings.TrimSpace(keyValue[1])
if strings.HasPrefix(value, "\"") && strings.HasSuffix(value, "\"") {
value = value[1 : len(value)-1]
}
tempMap[key] = value
}
//从 map 中提取值并填充到 struct
authParams := &BearerAuthParams{}
if realm, ok := tempMap["realm"]; ok {
authParams.Realm = realm
}
if service, ok := tempMap["service"]; ok {
authParams.Service = service
}
if scope, ok := tempMap["scope"]; ok {
authParams.Scope = scope
}
return authParams, nil
}

64
proxy/bandwidth.go Normal file
View File

@@ -0,0 +1,64 @@
package proxy
import (
"errors"
"ghproxy/config"
"github.com/WJQSERVER-STUDIO/go-utils/limitreader"
"golang.org/x/time/rate"
)
var (
bandwidthLimit rate.Limit
bandwidthBurst rate.Limit
)
func UnDefiendRateStringErrHandle(err error) error {
if errors.Is(err, &limitreader.UnDefiendRateStringErr{}) {
logWarning("UnDefiendRateStringErr: %s", err)
return nil
}
return err
}
func SetGlobalRateLimit(cfg *config.Config) error {
if cfg.RateLimit.BandwidthLimit.Enabled {
var err error
var totalLimit rate.Limit
var totalBurst rate.Limit
totalLimit, err = limitreader.ParseRate(cfg.RateLimit.BandwidthLimit.TotalLimit)
if UnDefiendRateStringErrHandle(err) != nil {
logError("Failed to parse total bandwidth limit: %v", err)
return err
}
totalBurst, err = limitreader.ParseRate(cfg.RateLimit.BandwidthLimit.TotalBurst)
if UnDefiendRateStringErrHandle(err) != nil {
logError("Failed to parse total bandwidth burst: %v", err)
return err
}
limitreader.SetGlobalRateLimit(totalLimit, int(totalBurst))
err = SetBandwidthLimit(cfg)
if UnDefiendRateStringErrHandle(err) != nil {
logError("Failed to set bandwidth limit: %v", err)
return err
}
} else {
limitreader.SetGlobalRateLimit(rate.Inf, 0)
}
return nil
}
func SetBandwidthLimit(cfg *config.Config) error {
var err error
bandwidthLimit, err = limitreader.ParseRate(cfg.RateLimit.BandwidthLimit.SingleLimit)
if UnDefiendRateStringErrHandle(err) != nil {
logError("Failed to parse bandwidth limit: %v", err)
return err
}
bandwidthBurst, err = limitreader.ParseRate(cfg.RateLimit.BandwidthLimit.SingleBurst)
if UnDefiendRateStringErrHandle(err) != nil {
logError("Failed to parse bandwidth burst: %v", err)
return err
}
return nil
}

View File

@@ -8,21 +8,34 @@ import (
"net/http"
"strconv"
"github.com/WJQSERVER-STUDIO/go-utils/limitreader"
"github.com/cloudwego/hertz/pkg/app"
)
func ChunkedProxyRequest(ctx context.Context, c *app.RequestContext, u string, cfg *config.Config, matcher string) {
var (
method []byte
req *http.Request
resp *http.Response
err error
req *http.Request
resp *http.Response
err error
)
method = c.Request.Method()
go func() {
<-ctx.Done()
if resp != nil && resp.Body != nil {
resp.Body.Close()
}
if req != nil {
req.Body.Close()
}
}()
req, err = client.NewRequest(string(method), u, c.Request.BodyStream())
rb := client.NewRequestBuilder(string(c.Request.Method()), u)
rb.NoDefaultHeaders()
rb.SetBody(c.Request.BodyStream())
rb.WithContext(ctx)
req, err = rb.Build()
if err != nil {
HandleError(c, fmt.Sprintf("Failed to create request: %v", err))
return
@@ -58,8 +71,7 @@ func ChunkedProxyRequest(ctx context.Context, c *app.RequestContext, u string, c
bodySize = -1
}
if err == nil && bodySize > sizelimit {
var finalURL string
finalURL = resp.Request.URL.String()
finalURL := resp.Request.URL.String()
err = resp.Body.Close()
if err != nil {
logError("Failed to close response body: %v", err)
@@ -92,6 +104,12 @@ func ChunkedProxyRequest(ctx context.Context, c *app.RequestContext, u string, c
c.Status(resp.StatusCode)
bodyReader := resp.Body
if cfg.RateLimit.BandwidthLimit.Enabled {
bodyReader = limitreader.NewRateLimitedReader(bodyReader, bandwidthLimit, int(bandwidthBurst), ctx)
}
if MatcherShell(u) && matchString(matcher, matchedMatchers) && cfg.Shell.Editor {
// 判断body是不是gzip
var compress string
@@ -99,24 +117,25 @@ func ChunkedProxyRequest(ctx context.Context, c *app.RequestContext, u string, c
compress = "gzip"
}
logDebug("Use Shell Editor: %s %s %s %s %s", c.ClientIP(), method, u, c.Request.Header.Get("User-Agent"), c.Request.Header.GetProtocol())
logDebug("Use Shell Editor: %s %s %s %s %s", c.ClientIP(), c.Request.Method(), u, c.Request.Header.Get("User-Agent"), c.Request.Header.GetProtocol())
c.Header("Content-Length", "")
var reader io.Reader
reader, _, err = processLinks(resp.Body, compress, string(c.Request.Host()), cfg)
reader, _, err = processLinks(bodyReader, compress, string(c.Request.Host()), cfg)
c.SetBodyStream(reader, -1)
if err != nil {
logError("%s %s %s %s %s Failed to copy response body: %v", c.ClientIP(), method, u, c.Request.Header.Get("User-Agent"), c.Request.Header.GetProtocol(), err)
logError("%s %s %s %s %s Failed to copy response body: %v", c.ClientIP(), c.Request.Method(), u, c.Request.Header.Get("User-Agent"), c.Request.Header.GetProtocol(), err)
ErrorPage(c, NewErrorWithStatusLookup(500, fmt.Sprintf("Failed to copy response body: %v", err)))
return
}
} else {
if contentLength != "" {
c.SetBodyStream(resp.Body, bodySize)
c.SetBodyStream(bodyReader, bodySize)
return
}
c.SetBodyStream(resp.Body, -1)
c.SetBodyStream(bodyReader, -1)
}
}

View File

@@ -2,33 +2,131 @@ package proxy
import (
"context"
"encoding/json"
"fmt"
"ghproxy/config"
"ghproxy/weakcache"
"io"
"net/http"
"strconv"
"strings"
"github.com/WJQSERVER-STUDIO/go-utils/limitreader"
"github.com/cloudwego/hertz/pkg/app"
)
var (
dockerhubTarget = "registry-1.docker.io"
ghcrTarget = "ghcr.io"
)
var cache *weakcache.Cache[string]
type imageInfo struct {
User string
Repo string
Image string
}
func InitWeakCache() *weakcache.Cache[string] {
cache = weakcache.NewCache[string](weakcache.DefaultExpiration, 100)
return cache
}
/*
func GhcrRouting(cfg *config.Config) app.HandlerFunc {
return func(ctx context.Context, c *app.RequestContext) {
if cfg.Docker.Enabled {
if cfg.Docker.Target == "ghcr" {
GhcrRequest(ctx, c, "https://ghcr.io"+string(c.Request.RequestURI()), cfg, "ghcr")
} else if cfg.Docker.Target == "dockerhub" {
GhcrRequest(ctx, c, "https://registry-1.docker.io"+string(c.Request.RequestURI()), cfg, "dockerhub")
charToFind := '.'
reqTarget := c.Param("target")
path := ""
target := ""
if strings.ContainsRune(reqTarget, charToFind) {
path = c.Param("filepath")
if reqTarget == "docker.io" {
target = dockerhubTarget
} else if reqTarget == "ghcr.io" {
target = ghcrTarget
} else {
ErrorPage(c, NewErrorWithStatusLookup(403, "Docker Target is not Allowed"))
return
target = reqTarget
}
} else {
ErrorPage(c, NewErrorWithStatusLookup(403, "Docker is not Allowed"))
return
path = string(c.Request.RequestURI())
}
GhcrToTarget(ctx, c, cfg, target, path, nil)
}
}
*/
func GhcrWithImageRouting(cfg *config.Config) app.HandlerFunc {
return func(ctx context.Context, c *app.RequestContext) {
charToFind := '.'
reqTarget := c.Param("target")
reqImageUser := c.Param("user")
reqImageName := c.Param("repo")
reqFilePath := c.Param("filepath")
path := fmt.Sprintf("%s/%s/%s", reqImageUser, reqImageName, reqFilePath)
target := ""
if strings.ContainsRune(reqTarget, charToFind) {
if reqTarget == "docker.io" {
target = dockerhubTarget
} else if reqTarget == "ghcr.io" {
target = ghcrTarget
} else {
target = reqTarget
}
} else {
path = string(c.Request.RequestURI())
reqImageUser = c.Param("target")
reqImageName = c.Param("user")
}
image := &imageInfo{
User: reqImageUser,
Repo: reqImageName,
Image: fmt.Sprintf("%s/%s", reqImageUser, reqImageName),
}
GhcrToTarget(ctx, c, cfg, target, path, image)
}
}
func GhcrToTarget(ctx context.Context, c *app.RequestContext, cfg *config.Config, target string, path string, image *imageInfo) {
if cfg.Docker.Enabled {
if target != "" {
GhcrRequest(ctx, c, "https://"+target+"/v2/"+path+"?"+string(c.Request.QueryString()), image, cfg, target)
} else {
if cfg.Docker.Target == "ghcr" {
GhcrRequest(ctx, c, "https://"+ghcrTarget+string(c.Request.RequestURI()), image, cfg, ghcrTarget)
} else if cfg.Docker.Target == "dockerhub" {
GhcrRequest(ctx, c, "https://"+dockerhubTarget+string(c.Request.RequestURI()), image, cfg, dockerhubTarget)
} else if cfg.Docker.Target != "" {
// 自定义taget
GhcrRequest(ctx, c, "https://"+cfg.Docker.Target+string(c.Request.RequestURI()), image, cfg, cfg.Docker.Target)
} else {
// 配置为空
ErrorPage(c, NewErrorWithStatusLookup(403, "Docker Target is not set"))
return
}
}
} else {
ErrorPage(c, NewErrorWithStatusLookup(403, "Docker is not Allowed"))
return
}
}
func GhcrRequest(ctx context.Context, c *app.RequestContext, u string, cfg *config.Config, matcher string) {
func GhcrRequest(ctx context.Context, c *app.RequestContext, u string, image *imageInfo, cfg *config.Config, target string) {
var (
method []byte
@@ -37,13 +135,23 @@ func GhcrRequest(ctx context.Context, c *app.RequestContext, u string, cfg *conf
err error
)
go func() {
<-ctx.Done()
if resp != nil && resp.Body != nil {
resp.Body.Close()
}
if req != nil {
req.Body.Close()
}
}()
method = c.Request.Method()
rb := client.NewRequestBuilder(string(method), u)
rb := ghcrclient.NewRequestBuilder(string(method), u)
rb.NoDefaultHeaders()
rb.SetBody(c.Request.BodyStream())
rb.WithContext(ctx)
//req, err = client.NewRequest(string(method), u, c.Request.BodyStream())
req, err = rb.Build()
if err != nil {
HandleError(c, fmt.Sprintf("Failed to create request: %v", err))
@@ -56,14 +164,68 @@ func GhcrRequest(ctx context.Context, c *app.RequestContext, u string, cfg *conf
req.Header.Add(headerKey, headerValue)
})
resp, err = client.Do(req)
req.Header.Set("Host", target)
if image != nil {
token, exist := cache.Get(image.Image)
if exist {
logDebug("Use Cache Token: %s", token)
req.Header.Set("Authorization", "Bearer "+token)
}
}
resp, err = ghcrclient.Do(req)
if err != nil {
HandleError(c, fmt.Sprintf("Failed to send request: %v", err))
return
}
// 错误处理(404)
if resp.StatusCode == 404 {
// 处理状态码
if resp.StatusCode == 401 {
// 请求target /v2/路径
if string(c.Request.URI().Path()) != "/v2/" {
resp.Body.Close()
if image == nil {
ErrorPage(c, NewErrorWithStatusLookup(401, "Unauthorized"))
return
}
token := ChallengeReq(target, image, ctx, c)
// 更新kv
if token != "" {
logDump("Update Cache Token: %s", token)
cache.Put(image.Image, token)
}
rb := ghcrclient.NewRequestBuilder(string(method), u)
rb.NoDefaultHeaders()
rb.SetBody(c.Request.BodyStream())
rb.WithContext(ctx)
req, err = rb.Build()
if err != nil {
HandleError(c, fmt.Sprintf("Failed to create request: %v", err))
return
}
c.Request.Header.VisitAll(func(key, value []byte) {
headerKey := string(key)
headerValue := string(value)
req.Header.Add(headerKey, headerValue)
})
req.Header.Set("Host", target)
if token != "" {
req.Header.Set("Authorization", "Bearer "+token)
}
resp, err = ghcrclient.Do(req)
if err != nil {
HandleError(c, fmt.Sprintf("Failed to send request: %v", err))
return
}
}
} else if resp.StatusCode == 404 { // 错误处理(404)
ErrorPage(c, NewErrorWithStatusLookup(404, "Page Not Found (From Github)"))
return
}
@@ -84,8 +246,7 @@ func GhcrRequest(ctx context.Context, c *app.RequestContext, u string, cfg *conf
bodySize = -1
}
if err == nil && bodySize > sizelimit {
var finalURL string
finalURL = resp.Request.URL.String()
finalURL := resp.Request.URL.String()
err = resp.Body.Close()
if err != nil {
logError("Failed to close response body: %v", err)
@@ -99,17 +260,97 @@ func GhcrRequest(ctx context.Context, c *app.RequestContext, u string, cfg *conf
// 复制响应头,排除需要移除的 header
for key, values := range resp.Header {
for _, value := range values {
//c.Header(key, value)
c.Response.Header.Add(key, value)
}
}
c.Status(resp.StatusCode)
bodyReader := resp.Body
if cfg.RateLimit.BandwidthLimit.Enabled {
bodyReader = limitreader.NewRateLimitedReader(bodyReader, bandwidthLimit, int(bandwidthBurst), ctx)
}
if contentLength != "" {
c.SetBodyStream(resp.Body, bodySize)
c.SetBodyStream(bodyReader, bodySize)
return
}
c.SetBodyStream(resp.Body, -1)
c.SetBodyStream(bodyReader, -1)
}
type AuthToken struct {
Token string `json:"token"`
}
func ChallengeReq(target string, image *imageInfo, ctx context.Context, c *app.RequestContext) (token string) {
var resp401 *http.Response
var req401 *http.Request
var err error
rb401 := ghcrclient.NewRequestBuilder("GET", "https://"+target+"/v2/")
rb401.NoDefaultHeaders()
rb401.WithContext(ctx)
rb401.AddHeader("User-Agent", "docker/28.1.1 go/go1.23.8 git-commit/01f442b kernel/6.12.25-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/28.1.1 ")
req401, err = rb401.Build()
if err != nil {
HandleError(c, fmt.Sprintf("Failed to create request: %v", err))
return
}
req401.Header.Set("Host", target)
resp401, err = ghcrclient.Do(req401)
if err != nil {
HandleError(c, fmt.Sprintf("Failed to send request: %v", err))
return
}
defer resp401.Body.Close()
bearer, err := parseBearerWWWAuthenticateHeader(resp401.Header.Get("Www-Authenticate"))
if err != nil {
logError("Failed to parse Www-Authenticate header: %v", err)
return
}
scope := fmt.Sprintf("repository:%s:pull", image.Image)
getAuthRB := ghcrclient.NewRequestBuilder("GET", bearer.Realm).
NoDefaultHeaders().
WithContext(ctx).
AddHeader("User-Agent", "docker/28.1.1 go/go1.23.8 git-commit/01f442b kernel/6.12.25-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/28.1.1 ").
SetHeader("Host", bearer.Service).
AddQueryParam("service", bearer.Service).
AddQueryParam("scope", scope)
getAuthReq, err := getAuthRB.Build()
if err != nil {
logError("Failed to create request: %v", err)
return
}
authResp, err := ghcrclient.Do(getAuthReq)
if err != nil {
logError("Failed to send request: %v", err)
return
}
defer authResp.Body.Close()
bodyBytes, err := io.ReadAll(authResp.Body)
if err != nil {
logError("Failed to read auth response body: %v", err)
return
}
// 解码json
var authToken AuthToken
err = json.Unmarshal(bodyBytes, &authToken)
if err != nil {
logError("Failed to decode auth response body: %v", err)
return
}
token = authToken.Token
return token
}

View File

@@ -8,15 +8,32 @@ import (
"net/http"
"strconv"
"github.com/WJQSERVER-STUDIO/go-utils/limitreader"
"github.com/cloudwego/hertz/pkg/app"
)
func GitReq(ctx context.Context, c *app.RequestContext, u string, cfg *config.Config, mode string) {
var (
req *http.Request
resp *http.Response
)
go func() {
<-ctx.Done()
if resp != nil && resp.Body != nil {
resp.Body.Close()
}
if req != nil {
req.Body.Close()
}
}()
method := string(c.Request.Method())
bodyReader := bytes.NewBuffer(c.Request.Body())
reqBodyReader := bytes.NewBuffer(c.Request.Body())
//bodyReader := c.Request.BodyStream()
//bodyReader := c.Request.BodyStream() // 不可替换为此实现
if cfg.GitClone.Mode == "cache" {
userPath, repoPath, remainingPath, queryParams, err := extractParts(u)
@@ -28,14 +45,11 @@ func GitReq(ctx context.Context, c *app.RequestContext, u string, cfg *config.Co
u = cfg.GitClone.SmartGitAddr + userPath + repoPath + remainingPath + "?" + queryParams.Encode()
}
var (
resp *http.Response
)
if cfg.GitClone.Mode == "cache" {
rb := gitclient.NewRequestBuilder(method, u)
rb.NoDefaultHeaders()
rb.SetBody(bodyReader)
rb.SetBody(reqBodyReader)
rb.WithContext(ctx)
req, err := rb.Build()
if err != nil {
@@ -54,7 +68,8 @@ func GitReq(ctx context.Context, c *app.RequestContext, u string, cfg *config.Co
} else {
rb := client.NewRequestBuilder(string(c.Request.Method()), u)
rb.NoDefaultHeaders()
rb.SetBody(bodyReader)
rb.SetBody(reqBodyReader)
rb.WithContext(ctx)
req, err := rb.Build()
if err != nil {
@@ -89,7 +104,6 @@ func GitReq(ctx context.Context, c *app.RequestContext, u string, cfg *config.Co
for key, values := range resp.Header {
for _, value := range values {
//c.Header(key, value)
c.Response.Header.Add(key, value)
}
}
@@ -122,5 +136,11 @@ func GitReq(ctx context.Context, c *app.RequestContext, u string, cfg *config.Co
c.Response.Header.Set("Expires", "0")
}
c.SetBodyStream(resp.Body, -1)
bodyReader := resp.Body
if cfg.RateLimit.BandwidthLimit.Enabled {
bodyReader = limitreader.NewRateLimitedReader(bodyReader, bandwidthLimit, int(bandwidthBurst), ctx)
}
c.SetBodyStream(bodyReader, -1)
}

View File

@@ -12,17 +12,26 @@ import (
var BufferSize int = 32 * 1024 // 32KB
var (
tr *http.Transport
gittr *http.Transport
client *httpc.Client
gitclient *httpc.Client
tr *http.Transport
gittr *http.Transport
client *httpc.Client
gitclient *httpc.Client
ghcrtr *http.Transport
ghcrclient *httpc.Client
)
func InitReq(cfg *config.Config) {
func InitReq(cfg *config.Config) error {
initHTTPClient(cfg)
if cfg.GitClone.Mode == "cache" {
initGitHTTPClient(cfg)
}
initGhcrHTTPClient(cfg)
err := SetGlobalRateLimit(cfg)
if err != nil {
return err
}
return nil
}
func initHTTPClient(cfg *config.Config) {
@@ -72,6 +81,7 @@ func initHTTPClient(cfg *config.Config) {
httpc.WithTransport(tr),
)
}
}
func initGitHTTPClient(cfg *config.Config) {
@@ -142,3 +152,51 @@ func initGitHTTPClient(cfg *config.Config) {
)
}
}
func initGhcrHTTPClient(cfg *config.Config) {
var proTolcols = new(http.Protocols)
proTolcols.SetHTTP1(true)
proTolcols.SetHTTP2(true)
if cfg.Httpc.Mode == "auto" {
ghcrtr = &http.Transport{
IdleConnTimeout: 30 * time.Second,
WriteBufferSize: 32 * 1024, // 32KB
ReadBufferSize: 32 * 1024, // 32KB
Protocols: proTolcols,
}
} else if cfg.Httpc.Mode == "advanced" {
ghcrtr = &http.Transport{
MaxIdleConns: cfg.Httpc.MaxIdleConns,
MaxConnsPerHost: cfg.Httpc.MaxConnsPerHost,
MaxIdleConnsPerHost: cfg.Httpc.MaxIdleConnsPerHost,
WriteBufferSize: 32 * 1024, // 32KB
ReadBufferSize: 32 * 1024, // 32KB
Protocols: proTolcols,
}
} else {
// 错误的模式
logError("unknown httpc mode: %s", cfg.Httpc.Mode)
fmt.Println("unknown httpc mode: ", cfg.Httpc.Mode)
logWarning("use Auto to Run HTTP Client")
fmt.Println("use Auto to Run HTTP Client")
ghcrtr = &http.Transport{
IdleConnTimeout: 30 * time.Second,
WriteBufferSize: 32 * 1024, // 32KB
ReadBufferSize: 32 * 1024, // 32KB
}
}
if cfg.Outbound.Enabled {
initTransport(cfg, ghcrtr)
}
if cfg.Server.Debug {
ghcrclient = httpc.New(
httpc.WithTransport(ghcrtr),
httpc.WithDumpLog(),
)
} else {
ghcrclient = httpc.New(
httpc.WithTransport(ghcrtr),
)
}
}

View File

@@ -1,11 +1,8 @@
package proxy
import (
"bufio"
"compress/gzip"
"fmt"
"ghproxy/config"
"io"
"net/url"
"regexp"
"strings"
@@ -20,9 +17,12 @@ func Matcher(rawPath string, cfg *config.Config) (string, string, string, *GHPro
// 匹配 "https://github.com"开头的链接
if strings.HasPrefix(rawPath, "https://github.com") {
remainingPath := strings.TrimPrefix(rawPath, "https://github.com")
if strings.HasPrefix(remainingPath, "/") {
remainingPath = strings.TrimPrefix(remainingPath, "/")
}
/*
if strings.HasPrefix(remainingPath, "/") {
remainingPath = strings.TrimPrefix(remainingPath, "/")
}
*/
remainingPath = strings.TrimPrefix(remainingPath, "/")
// 预期格式/user/repo/more...
// 取出user和repo和最后部分
parts := strings.Split(remainingPath, "/")
@@ -104,62 +104,6 @@ func Matcher(rawPath string, cfg *config.Config) (string, string, string, *GHPro
return "", "", "", NewErrorWithStatusLookup(404, errMsg)
}
func EditorMatcher(rawPath string, cfg *config.Config) (bool, error) {
// 匹配 "https://github.com"开头的链接
if strings.HasPrefix(rawPath, "https://github.com") {
return true, nil
}
// 匹配 "https://raw.githubusercontent.com"开头的链接
if strings.HasPrefix(rawPath, "https://raw.githubusercontent.com") {
return true, nil
}
// 匹配 "https://raw.github.com"开头的链接
if strings.HasPrefix(rawPath, "https://raw.github.com") {
return true, nil
}
// 匹配 "https://gist.githubusercontent.com"开头的链接
if strings.HasPrefix(rawPath, "https://gist.githubusercontent.com") {
return true, nil
}
// 匹配 "https://gist.github.com"开头的链接
if strings.HasPrefix(rawPath, "https://gist.github.com") {
return true, nil
}
if cfg.Shell.RewriteAPI {
// 匹配 "https://api.github.com/"开头的链接
if strings.HasPrefix(rawPath, "https://api.github.com") {
return true, nil
}
}
return false, nil
}
// 匹配文件扩展名是sh的rawPath
func MatcherShell(rawPath string) bool {
return strings.HasSuffix(rawPath, ".sh")
}
// LinkProcessor 是一个函数类型,用于处理提取到的链接。
type LinkProcessor func(string) string
// 自定义 URL 修改函数
func modifyURL(url string, host string, cfg *config.Config) string {
// 去除url内的https://或http://
matched, err := EditorMatcher(url, cfg)
if err != nil {
logDump("Invalid URL: %s", url)
return url
}
if matched {
var u = url
u = strings.TrimPrefix(u, "https://")
u = strings.TrimPrefix(u, "http://")
logDump("Modified URL: %s", "https://"+host+"/"+u)
return "https://" + host + "/" + u
}
return url
}
var (
matchedMatchers = []string{
"blob",
@@ -211,118 +155,3 @@ func extractParts(rawURL string) (string, string, string, url.Values, error) {
}
var urlPattern = regexp.MustCompile(`https?://[^\s'"]+`)
// processLinks 处理链接,返回包含处理后数据的 io.Reader
func processLinks(input io.ReadCloser, compress string, host string, cfg *config.Config) (readerOut io.Reader, written int64, err error) {
pipeReader, pipeWriter := io.Pipe() // 创建 io.Pipe
readerOut = pipeReader
go func() { // 在 Goroutine 中执行写入操作
defer func() {
if pipeWriter != nil { // 确保 pipeWriter 关闭,即使发生错误
if err != nil {
if closeErr := pipeWriter.CloseWithError(err); closeErr != nil { // 如果有错误,传递错误给 reader
logError("pipeWriter close with error failed: %v, original error: %v", closeErr, err)
}
} else {
if closeErr := pipeWriter.Close(); closeErr != nil { // 没有错误,正常关闭
logError("pipeWriter close failed: %v", closeErr)
if err == nil { // 如果之前没有错误,记录关闭错误
err = closeErr
}
}
}
}
}()
defer func() {
if err := input.Close(); err != nil {
logError("input close failed: %v", err)
}
}()
var bufReader *bufio.Reader
if compress == "gzip" {
// 解压gzip
gzipReader, gzipErr := gzip.NewReader(input)
if gzipErr != nil {
err = fmt.Errorf("gzip解压错误: %v", gzipErr)
return // Goroutine 中使用 return 返回错误
}
defer gzipReader.Close()
bufReader = bufio.NewReader(gzipReader)
} else {
bufReader = bufio.NewReader(input)
}
var bufWriter *bufio.Writer
var gzipWriter *gzip.Writer
// 根据是否gzip确定 writer 的创建
if compress == "gzip" {
gzipWriter = gzip.NewWriter(pipeWriter) // 使用 pipeWriter
bufWriter = bufio.NewWriterSize(gzipWriter, 4096) //设置缓冲区大小
} else {
bufWriter = bufio.NewWriterSize(pipeWriter, 4096) // 使用 pipeWriter
}
//确保writer关闭
defer func() {
var closeErr error // 局部变量用于保存defer中可能发生的错误
if gzipWriter != nil {
if closeErr = gzipWriter.Close(); closeErr != nil {
logError("gzipWriter close failed %v", closeErr)
// 如果已经存在错误,则保留。否则,记录此错误。
if err == nil {
err = closeErr
}
}
}
if flushErr := bufWriter.Flush(); flushErr != nil {
logError("writer flush failed %v", flushErr)
// 如果已经存在错误,则保留。否则,记录此错误。
if err == nil {
err = flushErr
}
}
}()
// 使用正则表达式匹配 http 和 https 链接
for {
line, readErr := bufReader.ReadString('\n')
if readErr != nil {
if readErr == io.EOF {
break // 文件结束
}
err = fmt.Errorf("读取行错误: %v", readErr) // 传递错误
return // Goroutine 中使用 return 返回错误
}
// 替换所有匹配的 URL
modifiedLine := urlPattern.ReplaceAllStringFunc(line, func(originalURL string) string {
logDump("originalURL: %s", originalURL)
return modifyURL(originalURL, host, cfg) // 假设 modifyURL 函数已定义
})
n, writeErr := bufWriter.WriteString(modifiedLine)
written += int64(n) // 更新写入的字节数
if writeErr != nil {
err = fmt.Errorf("写入文件错误: %v", writeErr) // 传递错误
return // Goroutine 中使用 return 返回错误
}
}
// 在返回之前,再刷新一次 (虽然 defer 中已经有 flush但这里再加一次确保及时刷新)
if flushErr := bufWriter.Flush(); flushErr != nil {
if err == nil { // 避免覆盖之前的错误
err = flushErr
}
return // Goroutine 中使用 return 返回错误
}
}()
return readerOut, written, nil // 返回 reader 和 writtenerror 由 Goroutine 通过 pipeWriter.CloseWithError 传递
}

185
proxy/nest.go Normal file
View File

@@ -0,0 +1,185 @@
// Copyright 2025 WJQSERVER, WJQSERVER-STUDIO. All rights reserved.
// 使用本源代码受 WSL 2.0(WJQserver Studio License v2.0)与MPL 2.0(Mozilla Public License v2.0)许可协议的约束
// 此段代码使用双重授权许可, 允许用户选择其中一种许可证
package proxy
import (
"bufio"
"compress/gzip"
"fmt"
"ghproxy/config"
"io"
"strings"
)
func EditorMatcher(rawPath string, cfg *config.Config) (bool, error) {
// 匹配 "https://github.com"开头的链接
if strings.HasPrefix(rawPath, "https://github.com") {
return true, nil
}
// 匹配 "https://raw.githubusercontent.com"开头的链接
if strings.HasPrefix(rawPath, "https://raw.githubusercontent.com") {
return true, nil
}
// 匹配 "https://raw.github.com"开头的链接
if strings.HasPrefix(rawPath, "https://raw.github.com") {
return true, nil
}
// 匹配 "https://gist.githubusercontent.com"开头的链接
if strings.HasPrefix(rawPath, "https://gist.githubusercontent.com") {
return true, nil
}
// 匹配 "https://gist.github.com"开头的链接
if strings.HasPrefix(rawPath, "https://gist.github.com") {
return true, nil
}
if cfg.Shell.RewriteAPI {
// 匹配 "https://api.github.com/"开头的链接
if strings.HasPrefix(rawPath, "https://api.github.com") {
return true, nil
}
}
return false, nil
}
// 匹配文件扩展名是sh的rawPath
func MatcherShell(rawPath string) bool {
return strings.HasSuffix(rawPath, ".sh")
}
// LinkProcessor 是一个函数类型,用于处理提取到的链接。
type LinkProcessor func(string) string
// 自定义 URL 修改函数
func modifyURL(url string, host string, cfg *config.Config) string {
// 去除url内的https://或http://
matched, err := EditorMatcher(url, cfg)
if err != nil {
logDump("Invalid URL: %s", url)
return url
}
if matched {
var u = url
u = strings.TrimPrefix(u, "https://")
u = strings.TrimPrefix(u, "http://")
logDump("Modified URL: %s", "https://"+host+"/"+u)
return "https://" + host + "/" + u
}
return url
}
// processLinks 处理链接,返回包含处理后数据的 io.Reader
func processLinks(input io.ReadCloser, compress string, host string, cfg *config.Config) (readerOut io.Reader, written int64, err error) {
pipeReader, pipeWriter := io.Pipe() // 创建 io.Pipe
readerOut = pipeReader
go func() { // 在 Goroutine 中执行写入操作
defer func() {
if pipeWriter != nil { // 确保 pipeWriter 关闭,即使发生错误
if err != nil {
if closeErr := pipeWriter.CloseWithError(err); closeErr != nil { // 如果有错误,传递错误给 reader
logError("pipeWriter close with error failed: %v, original error: %v", closeErr, err)
}
} else {
if closeErr := pipeWriter.Close(); closeErr != nil { // 没有错误,正常关闭
logError("pipeWriter close failed: %v", closeErr)
if err == nil { // 如果之前没有错误,记录关闭错误
err = closeErr
}
}
}
}
}()
defer func() {
if err := input.Close(); err != nil {
logError("input close failed: %v", err)
}
}()
var bufReader *bufio.Reader
if compress == "gzip" {
// 解压gzip
gzipReader, gzipErr := gzip.NewReader(input)
if gzipErr != nil {
err = fmt.Errorf("gzip解压错误: %v", gzipErr)
return // Goroutine 中使用 return 返回错误
}
defer gzipReader.Close()
bufReader = bufio.NewReader(gzipReader)
} else {
bufReader = bufio.NewReader(input)
}
var bufWriter *bufio.Writer
var gzipWriter *gzip.Writer
// 根据是否gzip确定 writer 的创建
if compress == "gzip" {
gzipWriter = gzip.NewWriter(pipeWriter) // 使用 pipeWriter
bufWriter = bufio.NewWriterSize(gzipWriter, 4096) //设置缓冲区大小
} else {
bufWriter = bufio.NewWriterSize(pipeWriter, 4096) // 使用 pipeWriter
}
//确保writer关闭
defer func() {
var closeErr error // 局部变量用于保存defer中可能发生的错误
if gzipWriter != nil {
if closeErr = gzipWriter.Close(); closeErr != nil {
logError("gzipWriter close failed %v", closeErr)
// 如果已经存在错误,则保留。否则,记录此错误。
if err == nil {
err = closeErr
}
}
}
if flushErr := bufWriter.Flush(); flushErr != nil {
logError("writer flush failed %v", flushErr)
// 如果已经存在错误,则保留。否则,记录此错误。
if err == nil {
err = flushErr
}
}
}()
// 使用正则表达式匹配 http 和 https 链接
for {
line, readErr := bufReader.ReadString('\n')
if readErr != nil {
if readErr == io.EOF {
break // 文件结束
}
err = fmt.Errorf("读取行错误: %v", readErr) // 传递错误
return // Goroutine 中使用 return 返回错误
}
// 替换所有匹配的 URL
modifiedLine := urlPattern.ReplaceAllStringFunc(line, func(originalURL string) string {
logDump("originalURL: %s", originalURL)
return modifyURL(originalURL, host, cfg) // 假设 modifyURL 函数已定义
})
n, writeErr := bufWriter.WriteString(modifiedLine)
written += int64(n) // 更新写入的字节数
if writeErr != nil {
err = fmt.Errorf("写入文件错误: %v", writeErr) // 传递错误
return // Goroutine 中使用 return 返回错误
}
}
// 在返回之前,再刷新一次 (虽然 defer 中已经有 flush但这里再加一次确保及时刷新)
if flushErr := bufWriter.Flush(); flushErr != nil {
if err == nil { // 避免覆盖之前的错误
err = flushErr
}
return // Goroutine 中使用 return 返回错误
}
}()
return readerOut, written, nil // 返回 reader 和 writtenerror 由 Goroutine 通过 pipeWriter.CloseWithError 传递
}

View File

@@ -13,8 +13,7 @@ func listCheck(cfg *config.Config, c *app.RequestContext, user string, repo stri
// 白名单检查
if cfg.Whitelist.Enabled {
var whitelist bool
whitelist = auth.CheckWhitelist(user, repo)
whitelist := auth.CheckWhitelist(user, repo)
if !whitelist {
ErrorPage(c, NewErrorWithStatusLookup(403, fmt.Sprintf("Whitelist Blocked repo: %s/%s", user, repo)))
logInfo("%s %s %s %s %s Whitelist Blocked repo: %s/%s", c.ClientIP(), c.Method(), rawPath, c.Request.Header.UserAgent(), c.Request.Header.GetProtocol(), user, repo)
@@ -24,8 +23,7 @@ func listCheck(cfg *config.Config, c *app.RequestContext, user string, repo stri
// 黑名单检查
if cfg.Blacklist.Enabled {
var blacklist bool
blacklist = auth.CheckBlacklist(user, repo)
blacklist := auth.CheckBlacklist(user, repo)
if blacklist {
ErrorPage(c, NewErrorWithStatusLookup(403, fmt.Sprintf("Blacklist Blocked repo: %s/%s", user, repo)))
logInfo("%s %s %s %s %s Blacklist Blocked repo: %s/%s", c.ClientIP(), c.Method(), rawPath, c.Request.Header.UserAgent(), c.Request.Header.GetProtocol(), user, repo)

258
weakcache/weakcache.go Normal file
View File

@@ -0,0 +1,258 @@
package weakcache
import (
"container/list"
"sync"
"time"
"weak" // Go 1.24 引入的 weak 包
)
// DefaultExpiration 默认过期时间,这里设置为 15 分钟。
// 这是一个导出的常量,方便用户使用包时引用默认值。
const DefaultExpiration = 5 * time.Minute
// cleanupInterval 是后台清理 Go routine 的扫描间隔,这里设置为 5 分钟。
// 这是一个内部常量,不导出。
const cleanupInterval = 2 * time.Minute
// cacheEntry 缓存项的内部结构。不导出。
type cacheEntry[T any] struct {
Value T
Expiration time.Time
key string // 存储key方便在list.Element中引用
}
// Cache 是一个基于 weak.Pointer, 带有过期和大小上限 (FIFO) 的泛型缓存。
// 这是一个导出的类型。
type Cache[T any] struct {
mu sync.RWMutex
// 修正缓存存储key -> weak.Pointer 到 cacheEntry 结构体 (而不是指向结构体的指针)
// weak.Make(*cacheEntry[T]) 返回 weak.Pointer[cacheEntry[T]]
data map[string]weak.Pointer[cacheEntry[T]]
// FIFO 链表:存储 key 的 list.Element
// 链表头部是最近放入的,尾部是最早放入的(最老的)
fifoList *list.List
// FIFO 元素的映射key -> *list.Element
fifoMap map[string]*list.Element
defaultExpiration time.Duration
maxSize int // 缓存最大容量0 表示无限制
stopCleanup chan struct{}
wg sync.WaitGroup // 用于等待清理 Go routine 退出
}
// NewCache 创建一个新的缓存实例。
// expiration: 新添加项的默认过期时间。如果为 0则使用 DefaultExpiration。
// maxSize: 缓存的最大容量0 表示无限制。当达到上限时,采用 FIFO 策略淘汰。
// 这是一个导出的构造函数。
func NewCache[T any](expiration time.Duration, maxSize int) *Cache[T] {
if expiration <= 0 {
expiration = DefaultExpiration
}
c := &Cache[T]{
// 修正:初始化 map值类型已修正
data: make(map[string]weak.Pointer[cacheEntry[T]]),
fifoList: list.New(),
fifoMap: make(map[string]*list.Element),
defaultExpiration: expiration,
maxSize: maxSize,
stopCleanup: make(chan struct{}),
}
// 启动后台清理 Go routine
c.wg.Add(1)
go c.cleanupLoop()
return c
}
// Put 将值放入缓存。如果 key 已存在,会更新其值和过期时间。
// 这是导出的方法。
func (c *Cache[T]) Put(key string, value T) {
c.mu.Lock()
defer c.mu.Unlock()
now := time.Now()
expiration := now.Add(c.defaultExpiration)
// 如果 key 已经存在,更新其值和过期时间。
// 在 FIFO 策略中, Put 更新不改变其在链表中的位置,除非旧的 entry 已经被 GC。
if elem, ok := c.fifoMap[key]; ok {
// 从 data map 中获取弱引用wp 的类型现在是 weak.Pointer[cacheEntry[T]]
if wp, dataOk := c.data[key]; dataOk {
// wp.Value() 返回 *cacheEntry[T] entry 的类型现在是 *cacheEntry[T]
entry := wp.Value()
if entry != nil {
// 旧的 cacheEntry 仍在内存中,直接更新
entry.Value = value
entry.Expiration = expiration
// 在严格 FIFO 中,更新不移动位置
return
}
// 如果 weak.Pointer.Value() 为 nil说明之前的 cacheEntry 已经被 GC 了
// 此时需要创建一个新的 entry并将其从旧位置移除再重新添加
c.fifoList.Remove(elem)
delete(c.fifoMap, key)
} else {
c.fifoList.Remove(elem)
delete(c.fifoMap, key)
}
}
// 新建缓存项 (注意这里是结构体值,而不是指针)
// weak.Make 接收的是指针 *T
entry := &cacheEntry[T]{ // 创建结构体指针
Value: value,
Expiration: expiration,
key: key, // 存储 key
}
// 将新的 *cacheEntry[T] 包装成 weak.Pointer[cacheEntry[T]] 存入 data map
// weak.Make(entry) 现在返回 weak.Pointer[cacheEntry[T]],类型匹配 data map 的值类型
c.data[key] = weak.Make(entry)
// 添加到 FIFO 链表头部 (最近放入/更新的在头部)
// PushFront 返回新的 list.Element
c.fifoMap[key] = c.fifoList.PushFront(key)
// 检查大小上限并进行淘汰 (淘汰尾部的最老项)
c.evictIfNeeded()
}
// Get 从缓存中获取值。返回获取到的值和是否存在/是否有效。
// 这是导出的方法。
func (c *Cache[T]) Get(key string) (T, bool) {
c.mu.RLock() // 先读锁
// 从 data map 中获取弱引用wp 的类型现在是 weak.Pointer[cacheEntry[T]]
wp, ok := c.data[key]
c.mu.RUnlock() // 立即释放读锁如果需要写操作removeEntry可以获得锁
var zero T // 零值
if !ok {
return zero, false
}
// 尝试获取实际的 cacheEntry 指针
// wp.Value() 返回 *cacheEntry[T] entry 的类型现在是 *cacheEntry[T]
entry := wp.Value()
if entry == nil {
// 对象已被GC回收需要清理此弱引用
c.removeEntry(key) // 内部会加写锁
return zero, false
}
// 检查过期时间 (通过 entry 指针访问字段)
if time.Now().After(entry.Expiration) {
// 逻辑上已过期
c.removeEntry(key) // 内部会加写锁
return zero, false
}
// 在 FIFO 缓存中Get 操作不改变项在链表中的位置
return entry.Value, true // 通过 entry 指针访问值字段
}
// removeEntry 从缓存中移除项。
// 这个方法是内部使用的,不导出。需要被调用者确保持有写锁,或者内部自己加锁。
// 考虑到 Get 和 cleanupLoop 可能会调用,让其内部自己加锁更安全。
func (c *Cache[T]) removeEntry(key string) {
c.mu.Lock()
defer c.mu.Unlock()
// 从 data map 中删除
delete(c.data, key)
// 从 FIFO 链表和 fifoMap 中删除
if elem, ok := c.fifoMap[key]; ok {
c.fifoList.Remove(elem)
delete(c.fifoMap, key)
}
}
// evictIfNeeded 检查是否需要淘汰最老FIFO 链表尾部)的项。
// 这个方法是内部使用的,不导出。必须在持有写锁的情况下调用。
func (c *Cache[T]) evictIfNeeded() {
if c.maxSize > 0 && c.fifoList.Len() > c.maxSize {
// 淘汰 FIFO 链表尾部的元素 (最老的)
oldest := c.fifoList.Back()
if oldest != nil {
keyToEvict := oldest.Value.(string) // 链表元素存储的是 key
c.fifoList.Remove(oldest)
delete(c.fifoMap, keyToEvict)
delete(c.data, keyToEvict) // 移除弱引用
}
}
}
// Size 返回当前缓存中的弱引用项数量。
// 注意:这个数量可能包含已被 GC 回收但尚未清理的项。
// 这是一个导出的方法。
func (c *Cache[T]) Size() int {
c.mu.RLock()
defer c.mu.RUnlock()
return len(c.data)
}
// cleanupLoop 后台清理 Go routine。不导出。
func (c *Cache[T]) cleanupLoop() {
defer c.wg.Done()
// 使用内部常量 cleanupInterval
ticker := time.NewTicker(cleanupInterval)
defer ticker.Stop()
for {
select {
case <-ticker.C:
c.cleanupExpiredAndGCed()
case <-c.stopCleanup:
return
}
}
}
// cleanupExpiredAndGCed 扫描并清理已过期或已被 GC 回收的项。不导出。
func (c *Cache[T]) cleanupExpiredAndGCed() {
c.mu.Lock() // 清理时需要写锁
defer c.mu.Unlock()
now := time.Now()
keysToRemove := make([]string, 0, len(c.data)) // 预估容量
// 遍历 data map 查找需要清理的键
for key, wp := range c.data {
// wp 的类型是 weak.Pointer[cacheEntry[T]]
// wp.Value() 返回 *cacheEntry[T] entry 的类型是 *cacheEntry[T]
entry := wp.Value() // 尝试获取强引用
if entry == nil {
// 已被 GC 回收
keysToRemove = append(keysToRemove, key)
} else if now.After(entry.Expiration) {
// 逻辑过期 (通过 entry 指针访问字段)
keysToRemove = append(keysToRemove, key)
}
}
// 执行删除操作
for _, key := range keysToRemove {
// 从 data map 中删除
delete(c.data, key)
// 从 FIFO 链表和 fifoMap 中删除
// 需要再次检查 fifoMap因为在持有锁期间evictIfNeeded 可能已经移除了这个 key
if elem, ok := c.fifoMap[key]; ok {
c.fifoList.Remove(elem)
delete(c.fifoMap, key)
}
}
}
// StopCleanup 停止后台清理 Go routine。
// 这是一个导出的方法。
func (c *Cache[T]) StopCleanup() {
close(c.stopCleanup)
c.wg.Wait() // 等待 Go routine 退出
}